Storage space management methods, products, equipment and media

Abstract

The present invention discloses a storage space management method, product, device and medium, which relates to the field of distributed storage technology and is applied to the central node of a distributed storage system, including: obtaining a storage space allocation request sent by a client; sending a target space block corresponding to the storage space allocation request to the client, so that the client uses the target space block to record a target file and corresponding file header information; wherein the file header information includes a space mapping diagram for recording a mapping relationship between a target file and a target space block; receiving a storage space release request and a to-be-deleted space block sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the to-be-deleted space block is a space block determined based on the space mapping diagram of the target file; and releasing the to-be-deleted space block according to the storage space release request. Through the above scheme, the complexity of the storage space management process is reduced when only file storage is required.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to storage space management methods, products, equipment and media. Background Art

[0002] Ceph is an open source distributed storage system designed to provide high-performance, reliable and scalable storage solutions. Ceph can provide object storage, block storage and file system storage services at the same time, which makes it a very flexible choice for a variety of application scenarios, including cloud computing, data centers, large-scale data storage, etc. In the Ceph distributed storage system, OSD (Object Storage Daemon) is the key component responsible for managing the actual data storage. Each OSD node is responsible for managing a part of the storage space, and works together through the RADOS (Redundant Array of Independent Disks Object Storage) layer to ensure data distribution, replication and recovery functions.

[0003] OSD manages storage space and operates across the entire Ceph cluster, which means it can implement a wide range of functions, including object storage, block storage, and file storage. However, in some scenarios, users may only use file storage and do not need object storage. In this case, if OSD management is still used, it must be combined with RADOS components, resulting in the need for multiple layers of processing for the entire environment. In addition, file storage still needs to provide MDS (Metadata Server), which makes the current storage process too complicated.

[0004] It can be seen that how to reduce the complexity of the storage space management process when only file storage is required is a problem that those skilled in the art need to solve. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a storage space management method, device, equipment and medium to reduce the complexity of the storage space management process when only file storage is required. The specific scheme is as follows:

[0006] In a first aspect, the present invention discloses a storage space management method, which is applied to a central node of a distributed storage system, comprising:

[0007] Get the storage space allocation request sent by the client;

[0008] Sending the target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block;

[0009] Receive a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which a target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file;

[0010] The space block to be deleted is released according to the storage space release request.

[0011] Optionally, sending the target space block corresponding to the storage space allocation request to the client includes:

[0012] Determine candidate space blocks whose space block states are unallocated from each storage space block;

[0013] Determine a target space block corresponding to the storage space allocation request from each candidate space block, and set the state of the target space block to the allocation request state;

[0014] Send the target spatial block to the client.

[0015] Optionally, after sending the target space block to the client, the following is further included:

[0016] Change the state of the target space block from the allocation request state to the allocation confirmation state;

[0017] Obtain the reception confirmation message returned by the client based on the target space block;

[0018] The state of the target space block is changed from an allocation confirmed state to an allocated state based on receiving the confirmation message.

[0019] Optionally, determining a target space block corresponding to the storage space allocation request from each candidate space block includes:

[0020] Determine a requested allocated space size corresponding to the storage space allocation request, and determine a target space size based on the requested allocated space size; wherein the target space size is greater than the requested allocated space size;

[0021] A target spatial block corresponding to the target spatial size is determined from each candidate spatial block.

[0022] Optionally, send the target spatial block to the client, including:

[0023] Send the target space block to the client's space allocator;

[0024] Accordingly, the client uses the target space block to record the target file and the corresponding file header information, including:

[0025] The client obtains a target subspace block corresponding to the requested allocated space size from the target space block stored in its own space allocator, and uses the target subspace block to record the target file and the corresponding file header information.

[0026] Optionally, the client uses the target space block to record the target file and corresponding file header information, including:

[0027] The client determines the file header storage space from the target space block, generates the file header information of the target file including the size information of the target file, the file modification time, and the space mapping diagram, and saves the file header information to the file header storage space in the target space block.

[0028] Optionally, the file header storage space includes a mapping storage space and an information storage space;

[0029] Accordingly, the file header information is saved to the file header storage space in the target space block, including:

[0030] The space mapping map in the file header information is saved in the mapping map storage space;

[0031] The size information and file modification time in the file header information are saved to the information storage space.

[0032] Optionally, the space map in the file header information is saved to the map storage space, including:

[0033] Determine whether the size of the spatial map is greater than the size of the map storage space;

[0034] If the size of the spatial map is not greater than the size of the map storage space, the spatial map is saved in the map storage space;

[0035] If the size of the spatial mapping graph is larger than the size of the mapping graph storage space, the spatial mapping graph is split into multiple spatial mapping sub-graphs, and the target mapping sub-graphs in each spatial mapping sub-graph are saved in the mapping graph storage space based on a linked list mechanism.

[0036] Optionally, the space block to be deleted is released according to the storage space release request, including:

[0037] The space block information of the space block to be deleted is stored in the space release mapping table according to the storage space release request; wherein the space block information of the space block to be deleted includes an identifier of the space block to be deleted, a size of the space block to be deleted, and a state of the space block to be deleted;

[0038] Changing the state of the to-be-deleted space block to a release request receiving state, generating a request receiving record, and persistently storing the request receiving record in a first preset disk;

[0039] Send a release confirmation request to the target non-central node and change the state of the space block to be deleted to the release pending confirmation state;

[0040] Receiving a release reconfirmation request returned by the target non-central node based on the release confirmation request;

[0041] Based on the release reconfirmation request, the space block to be deleted is released, and the state of the space block to be deleted is changed to unallocated state.

[0042] Optionally, after sending the release confirmation request to the target non-central node, the process further includes:

[0043] Based on the release confirmation request, the target non-central node changes the state of the space block to be deleted to the client deletion release record state, deletes the space block to be deleted, generates a space block deletion record, and persistently stores the space block deletion record in the second preset disk.

[0044] Optionally, before receiving the storage space release request and the space block to be deleted sent by the target non-central node, the method further includes:

[0045] The space mapping diagram of the target file is read through the target non-central node to determine the space block to be deleted, the state of the space block to be deleted is set to a released state, and a space block to be deleted record is generated, and the space block to be deleted record is saved to a release tracking table in a third preset disk.

[0046] Optionally, the storage space management method further includes:

[0047] The storage space release request and the space block to be deleted are sent to the central node through the target non-central node, and the status of the space block to be deleted is set to the release request status sent by the client.

[0048] In a second aspect, the present invention discloses a computer program product, including a computer program / instruction, which implements the steps of the aforementioned storage space management method when executed by a processor.

[0049] In a third aspect, the present invention discloses an electronic device, comprising:

[0050] Memory, used to store computer programs;

[0051] The processor is used to execute a computer program to implement the steps of the storage space management method disclosed above.

[0052] In a fourth aspect, the present invention discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the storage space management method disclosed above are implemented.

[0053] It can be seen that the present invention is applied to the central node of a distributed storage system, including: obtaining a storage space allocation request sent by a client; sending a target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; receiving a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on the space mapping diagram of the target file; and releasing space for the space block to be deleted according to the storage space release request.

[0054] The beneficial effect is as follows: the present application allocates and releases storage space by the central node of the distributed storage system, that is, the central node receives the storage space allocation request sent by the client, and sends the target space block of the storage space allocation request to the client, so that the client uses the target space block to record the target file and the corresponding file header information, wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; further, when the target file needs to be deleted, the target non-central node to which the target file belongs can directly determine the corresponding space block to be deleted based on the space mapping diagram of the target file, and then the target non-central node sends the storage space release request and the space block to be deleted to the central node, and the central node can release the space of the space block to be deleted according to the storage space release request. It can be seen that when facing file storage scenarios, the present application can complete storage space allocation and release without relying on OSD and RADOS, effectively reducing the complexity of the storage space management process. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A flow chart of a storage space management method provided by an embodiment of the present invention;

[0057] Figure 2 A specific storage space allocation diagram provided by an embodiment of the present invention;

[0058] Figure 3 A specific schematic diagram of storage space release provided by an embodiment of the present invention;

[0059] Figure 4 A flow chart of a specific storage space management method provided by an embodiment of the present invention;

[0060] Figure 5 A specific schematic diagram of saving a spatial mapping diagram provided by an embodiment of the present invention;

[0061] Figure 6 A schematic diagram of the structure of a storage space management device provided by an embodiment of the present invention;

[0062] Figure 7 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Ceph is an open source distributed storage system designed to provide high-performance, reliable and scalable storage solutions. Ceph can provide object storage, block storage and file system storage services at the same time, which makes it a very flexible choice for a variety of application scenarios, including cloud computing, data centers, large-scale data storage, etc. In the Ceph distributed storage system, OSD is the key component responsible for managing the actual data storage. Each OSD node is responsible for managing a part of the storage space, and works together through the RADOS layer to ensure data distribution, replication and recovery functions.

[0065] OSD manages storage space and operates across the entire Ceph cluster, which means it can implement comprehensive functions such as object storage, block storage, and file storage. However, in some scenarios, users may only use file storage and do not need object storage. In this case, if OSD management is still used, it must be combined with RADOS components, resulting in the need for multiple layers to handle the entire environment. In addition, file storage still needs to provide MDS, which makes the current storage process too complicated.

[0066] The terms "including" and "having" in the specification of the present invention and the above-mentioned drawings, as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may include steps or units that are not listed.

[0067] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0068] Next, a storage space management solution provided by an embodiment of the present invention is introduced in detail. Figure 1 A storage space management method provided by an embodiment of the present invention is applied to a central node of a distributed storage system, comprising:

[0069] Step S11: Obtain a storage space allocation request sent by the client.

[0070] The distributed storage system includes multiple MDS nodes, and the central node and non-central node are determined from each MDS node. The MDS node can also be responsible for managing the metadata of the Ceph file system, that is, the directory tree structure and attribute information of the file system. The MDS nodes provide metadata services through mutual cooperation to support efficient access to large-scale file systems. The client accesses the storage space block and writes data to the storage space block, which requires the allocation and release of disk space corresponding to the storage space block to ensure the reading and writing of the client.

[0071] Step S12: Send the target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block.

[0072] In this embodiment, the target space block corresponding to the storage space allocation request is sent to the client, including: determining each candidate space block whose space block status is an unallocated state from each storage space block; determining the target space block corresponding to the storage space allocation request from each candidate space block, and setting the status of the target space block to an allocation request status; and sending the target space block to the client.

[0073] The distributed storage system contains multiple storage space blocks, such as Figure 2 A specific storage space allocation diagram is shown. When the central node receives a storage space allocation request sent by a client, it first determines candidate space blocks whose space block status is unallocated from each storage space block, and then determines the target space block corresponding to the storage space allocation request from each candidate space block.

[0074] In this embodiment, after the target space block is sent to the client, it also includes: changing the state of the target space block from an allocation request state to an allocation confirmation state; obtaining a reception confirmation message returned by the client based on the target space block; and changing the state of the target space block from an allocation confirmation state to an allocated state based on the reception confirmation message.

[0075] For example Figure 2 As shown, after the central node sends the target space block to the client, it changes the state of the target space block from the allocation request state to the allocation confirmation state. After the client receives the target space block, it sends a reception confirmation message to the central node to inform the central node that the client has received the target space block. Then the central node can change the state of the target space block from the allocation confirmation state to the allocated state. In this way, according to the state of the space block, it can be determined whether the space block is allocated and the allocation process stage. At this time, even if the central node hangs up, it can be determined that the target space block has been allocated after subsequent recovery, avoiding the space block from being allocated to different clients at the same time.

[0076] In a specific embodiment of the first target space block size determination, the target space block corresponding to the storage space allocation request is determined from each candidate space block, including: determining the requested allocation space size corresponding to the storage space allocation request, and determining the target space size based on the requested allocation space size; wherein the target space size is larger than the requested allocation space size; and determining the target space block corresponding to the target space size from each candidate space block. It can be understood that the target space size of the target space block is larger than the requested allocation space size corresponding to the storage space allocation request. For example, if the requested allocation space size corresponding to the storage space allocation request sent by the client is 1M, then the target space size can be determined to be 1G, and then a target space block of 1G is determined from each candidate space block.

[0077] In a specific embodiment of the second target space block size determination, the client determines the requested allocation space size corresponding to the storage space allocation request based on the size of the target file, and determines the target space size based on the requested allocation space size, wherein the requested allocation space size is equal to the target space size and greater than the size of the target file; and determines the target space block corresponding to the target space size from each candidate space block. That is, if the size of the target file is 1M, the requested allocation space size can be determined to be 1G, that is, a 1G target space block is determined from each candidate space block.

[0078] In this embodiment, sending the target space block to the client includes: sending the target space block to the space allocator of the client. It can be understood that after determining the target space block of 1G, the target space block of 1G is sent to the space allocator of the client.

[0079] According to the specific embodiment of the first target space block size determination, the specific embodiment of the client recording the target file is: the client uses the target space block to record the target file and the corresponding file header information, including: the client obtains the target subspace block corresponding to the requested allocated space size from the target space block stored in its own space allocator, and uses the target subspace block to record the target file and the corresponding file header information. After the central node sends the target space block to the client's space allocator, the client obtains the target subspace block corresponding to the requested allocated space size from the target space block stored in its own space allocator. It can be understood that because the requested allocated space size is smaller than the target space size of the target space block, the client can obtain the target subspace block from the space allocator multiple times, thereby reducing the frequency of the client sending storage space allocation requests to the central node.

[0080] According to the second specific embodiment of determining the target space block size, the specific embodiment of the client recording the target file is: the client obtains the target subspace block corresponding to the target file size from the target space block stored in its own space allocator. Because the target file size is smaller than the target space size of the target space block, the client can also obtain the target subspace block from the space allocator multiple times.

[0081] Step S13: receiving a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file.

[0082] In this embodiment, before receiving the storage space release request and the space block to be deleted sent by the target non-central node, it also includes: reading the space mapping map of the target file through the target non-central node to determine the space block to be deleted, setting the state of the space block to be deleted to a released state, and generating a space block to be deleted record, and saving the space block to be deleted record to the release tracking table in the third preset disk.

[0083] When the target file needs to be deleted, the space block used to record the target file needs to be released, that is, the space block recording the target file becomes the space block to be deleted, and the non-central node to which the target file belongs is the target non-central node. The target non-central node reads the space mapping map of the target file to determine the space block recording the target file, that is, the target non-central node reads the space mapping map of the target file to determine the space block to be deleted; further, the target non-central node sets the state of the space block to be deleted to the released state (that is, release_client_store), and generates a space block to be deleted record, and saves the space block to be deleted record to the release tracking table (that is, release_map) in the third preset disk. In this way, even if the target non-central node loses power at this time, this part of the space has been recorded and there is no need to worry about loss.

[0084] In this embodiment, it also includes: sending a storage space release request and a space block to be deleted to the central node through the target non-central node, and setting the state of the space block to be deleted to the client sending a release request state. The target non-central node sends a storage space release request (i.e., release_request) and a space block to be deleted to the central node, and sets the state of the space block to be deleted to the client sending a release request state (release_client_send).

[0085] Step S14: releasing the space block to be deleted according to the storage space release request.

[0086] In this embodiment, space is released for a space block to be deleted according to a storage space release request, including: saving space block information of the space block to be deleted in a space release mapping table according to the storage space release request; wherein the space block information of the space block to be deleted includes an identifier of the space block to be deleted, a size of the space block to be deleted, and a status of the space block to be deleted; changing the status of the space block to be deleted to a release request reception status, generating a request reception record, and persistently storing the request reception record in a first preset disk; sending a release confirmation request to a target non-central node, and changing the status of the space block to be deleted to a release pending confirmation status; receiving a release reconfirmation request returned by the target non-central node based on the release confirmation request; releasing space for the space block to be deleted based on the release reconfirmation request, and changing the status of the space block to be deleted to an unallocated status.

[0087] The central node saves the space block information of the space block to be deleted in the space release mapping table according to the storage space release request, wherein the space block information of the space block to be deleted includes the identifier of the space block to be deleted, the size of the space block to be deleted and the status of the space block to be deleted, and then changes the status of the space block to be deleted to the release request reception status (i.e., release_server_stored), and generates a request received record (i.e., server_release_map), and persistently stores the request received record in the first preset disk. Even if the central node and the target non-central node hang up at this time, it will not be affected, so as to indicate that the central node has received this part of the space block. Even if the subsequent target non-central node sends the same space block again, the central node finds from the request received record that the space block to be deleted already exists, and no processing is required, thereby ensuring that this part of the space will not be released repeatedly.

[0088] The central node sends a release confirmation request (i.e., release_request_ack) to the target non-central node, and changes the status of the space block to be deleted to a release pending confirmation status (i.e., release_server_ack); receives a release reconfirmation request (i.e., release_ack_ack) returned by the target non-central node based on the release confirmation request; releases the space block to be deleted based on the release reconfirmation request, and changes the status of the space block to be deleted to an unallocated status (i.e., release_server_release).

[0089] In this embodiment, after sending a release confirmation request to the target non-central node, it also includes: changing the state of the space block to be deleted to the client deletion release record state based on the release confirmation request through the target non-central node, deleting the space block to be deleted, and generating a space block deletion record, and persistently storing the space block deletion record in the second preset disk.

[0090] For example Figure 3A specific storage space release diagram is shown. The target non-central node receives a release confirmation request (i.e., release_request_ack), indicating that the central node has received the part of the space block. At this time, the target non-central node can delete the part of the space block. However, if the target non-central node loses power after the central node sends the request, the central node does not perceive whether the target non-central node has received the request. If the central node rashly releases the space block, the target non-central node does not know that it is likely to be sent repeatedly, resulting in repeated release by the central node. Therefore, in this embodiment, the target non-central node changes the state of the space block to be deleted to the client deletion release record state (i.e., release_client_delete) based on the release confirmation request (i.e., release_request_ack), deletes the space block to be deleted, generates a space block deletion record, and persistently stores the space block deletion record in the second preset disk. In this way, even if the target non-central node is restored after hanging up, the central node will not be asked to release the same space block repeatedly according to the space block deletion record.

[0091] Applying for space blocks and releasing space blocks are both done through requests, that is, through message transmission. If the central node fails and restarts, or other clients or target non-central nodes fail, the space allocated by the central node will not be actually delivered to the client, resulting in that part of the space is allocated but still cannot be used, or the released space cannot be delivered to the central node, resulting in the space that is considered to be released not being released, and ultimately insufficient space. Therefore, in this embodiment, multiple confirmations are required in the interaction process, and different states are added for each space block. These states are used to mark whether the space has been applied for or released, and each message interaction logic is finalized through three interactions.

[0092] It can be seen that the present invention is applied to the central node of a distributed storage system, including: obtaining a storage space allocation request sent by a client; sending a target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; receiving a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on the space mapping diagram of the target file; and releasing space for the space block to be deleted according to the storage space release request.

[0093] The beneficial effect is as follows: the present application allocates and releases storage space by the central node of the distributed storage system, that is, the central node receives the storage space allocation request sent by the client, and sends the target space block of the storage space allocation request to the client, so that the client uses the target space block to record the target file and the corresponding file header information, wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; further, when the target file needs to be deleted, the target non-central node to which the target file belongs can directly determine the corresponding space block to be deleted based on the space mapping diagram of the target file, and then the target non-central node sends the storage space release request and the space block to be deleted to the central node, and the central node can release the space of the space block to be deleted according to the storage space release request. It can be seen that when facing file storage scenarios, the present application can complete storage space allocation and release without relying on OSD and RADOS, effectively reducing the complexity of the storage space management process.

[0094] See also Figure 4 The embodiment of the present invention discloses a specific storage space management method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. The central node applied to the distributed storage system includes:

[0095] Step S21: Obtain a storage space allocation request sent by the client.

[0096] When the distributed storage system starts, the device blocks specified by the design are loaded to form a data pool and a metadata pool, and the central node is determined. The central node and the non-central node can be determined by analyzing the system log. Specifically: the system log and error report corresponding to each node in the distributed storage system are obtained, and the operation status of each node is determined according to the system log and error report, wherein the operation status is any one of the online state, the offline state, and the fault state; the first candidate nodes with the online operation status are screened out from each node; the network configuration information of each first candidate node is obtained by using a network diagnostic tool, wherein the network configuration information includes network interface and routing setting information; the second candidate nodes that meet the preset network reachability conditions are screened out from each first candidate node according to the network configuration information; the current load of each second candidate node is determined, and the central node with the smallest current load is screened out, and the nodes other than the central node are determined as non-central nodes.

[0097] Based on system logs and error reports, which are usually automatically generated by the system and can reflect the running status of the nodes in real time, by analyzing these logs and reports, the online, offline or fault status of each node can be accurately obtained, thereby ensuring the accuracy of the data. The entire determination process relies on objective data such as system logs, error reports and network diagnostic tools, reducing the interference of human factors, which makes the process of determining central nodes and non-central nodes more fair and transparent, avoiding subjective assumptions and bias. Through a series of automated steps to quickly determine central nodes and non-central nodes, this greatly improves the efficiency of the determination process and reduces the time cost of manual troubleshooting and confirmation. By obtaining the network interface and routing setting information of each candidate node through the network diagnostic tool, the network reachability of the node can be verified, which ensures that the selected central node is accessible on the network, thereby ensuring the connectivity and stability of the system. When determining the central node, the current load of each candidate node is taken into account, and the node with the smallest load is selected as the central node, which helps to optimize the load balancing of the system and improve the overall performance and response time of the system. This embodiment is not only applicable to determining the current central node and non-central node, but also can be dynamically adjusted according to changes in the system operation status. For example, when a node fails or the load is too high, the central node can be adjusted in time to ensure the continuous and stable operation of the system.

[0098] Step S22: Send the target space block corresponding to the storage space allocation request to the client, so that the client can determine the file header storage space from the target space block, generate file header information of the target file including the size information of the target file, the file modification time, and the space mapping map, and save the file header information to the file header storage space in the target space block; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block.

[0099] The client determines the file header storage space from the target space block. The file header storage space can be 4k in size, and generates the file header information of the target file including the size information of the target file, the file modification time, and the space mapping map. Each file cannot determine its own final size, so a map is used to save the corresponding space block requested. When writing data to the target file later, for example, when the target file appends 1M data, the file only has a 4k header, so it is necessary to apply for more than 1M of space from the central node again to store the data to be written by the client. At this time, this part of the newly allocated space is written to the corresponding space map. When reading and writing files subsequently, the corresponding space block can be found in the space map for reading and writing, thereby realizing the client's file reading and writing.

[0100] In this embodiment, the file header storage space includes a mapping storage space and an information storage space; accordingly, the file header information is saved to the file header storage space in the target space block, including: saving the space mapping in the file header information to the mapping storage space; saving the size information and file modification time in the file header information to the information storage space. Specifically, the size of the file header storage space is 4k, the size of the mapping storage space is 3k, and the size of the information storage space is 1k. The space mapping in the file header information is saved to the mapping storage space, and the size information and file modification time in the file header information are saved to the information storage space.

[0101] In this embodiment, the spatial mapping map in the file header information is saved in the mapping map storage space, including: determining whether the size of the spatial mapping map is larger than the space size of the mapping map storage space; if the size of the spatial mapping map is not larger than the space size of the mapping map storage space, saving the spatial mapping map in the mapping map storage space; if the size of the spatial mapping map is larger than the space size of the mapping map storage space, splitting the spatial mapping map into multiple spatial mapping sub-maps, and saving the target mapping sub-map in each spatial mapping sub-map in the mapping map storage space based on a linked list mechanism.

[0102] It can be understood that the spatial size of the spatial mapping map is proportional to the size of the target file. When the target file is too large, the spatial size of the spatial mapping map may exceed 3k. At this time, it is necessary to use the linked list mechanism to realize the storage of the spatial mapping map. Specifically, it is determined whether the size of the spatial mapping map is larger than the spatial size of the mapping map storage space, that is, whether the size of the spatial mapping map exceeds 3k; if the size of the spatial mapping map is not larger than the spatial size of the mapping map storage space, the spatial mapping map is saved in the mapping map storage space. If the size of the spatial mapping map is larger than the spatial size of the mapping map storage space, that is, the size of the spatial mapping map exceeds 3k, the spatial mapping map is split into multiple spatial mapping sub-maps, and the target mapping sub-maps in each spatial mapping sub-map are saved in the mapping map storage space based on the linked list mechanism. For example Figure 5 A specific spatial mapping map saving diagram is shown. When the spatial mapping map exceeds 3k, the spatial mapping map is split into multiple spatial mapping sub-maps, and the first spatial mapping sub-map is saved in the mapping map storage space, and a sub-mapping map storage space is reserved. The space is used to store the storage location information of the second spatial mapping sub-map, and the second spatial mapping sub-map is saved in the space block corresponding to the storage location information, and so on. Therefore, each target file only needs to save a 4K header, and the subsequent additional space needs to be read from the map in the header, and there is no need to append all the space to the file record.

[0103] Step S23: receiving a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file.

[0104] Step S24: releasing the space block to be deleted according to the storage space release request.

[0105] It can be seen that this application implements the storage of file space through 4k file header information and the corresponding space mapping linked list mechanism, and then implements the space application and release of the data pool and metadata pool through mds. After the client uses the request to apply for the space, it is handed over to its own space allocator for unified management. Mds then sends the released space to the fixed central node through a message request. In order to ensure the consistency of space application and release, the uniqueness of space management is achieved through three message interactions to ensure that there will be no repeated application and release, and different states are added for each space block to avoid releasing the wrong space block.

[0106] Figure 6 A schematic diagram of the structure of a storage space management device provided by an embodiment of the present invention, which is applied to a central node of a distributed storage system, includes:

[0107] A first request receiving module 11 is used to obtain a storage space allocation request sent by a client;

[0108] The space allocation module 12 is used to send the target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block;

[0109] The second request receiving module 13 is used to receive a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file;

[0110] The space release module 14 is used to release the space block to be deleted according to the storage space release request.

[0111] It can be seen that the present invention is applied to the central node of a distributed storage system, including: obtaining a storage space allocation request sent by a client; sending a target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; receiving a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on the space mapping diagram of the target file; and releasing space for the space block to be deleted according to the storage space release request.

[0112] The beneficial effect is as follows: the present application allocates and releases storage space by the central node of the distributed storage system, that is, the central node receives the storage space allocation request sent by the client, and sends the target space block of the storage space allocation request to the client, so that the client uses the target space block to record the target file and the corresponding file header information, wherein the file header information includes a space mapping diagram for recording the mapping relationship between the target file and the target space block; further, when the target file needs to be deleted, the target non-central node to which the target file belongs can directly determine the corresponding space block to be deleted based on the space mapping diagram of the target file, and then the target non-central node sends the storage space release request and the space block to be deleted to the central node, and the central node can release the space of the space block to be deleted according to the storage space release request. It can be seen that when facing file storage scenarios, the present application can complete storage space allocation and release without relying on OSD and RADOS, effectively reducing the complexity of the storage space management process.

[0113] Furthermore, the present application also discloses an electronic device. Figure 7 This is a block diagram of an electronic device according to an exemplary embodiment, and the content of the diagram cannot be considered as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the following steps:

[0114] Get the storage space allocation request sent by the client;

[0115] Sending the target space block corresponding to the storage space allocation request to the client, so that the client can use the target space block to record the target file and the corresponding file header information; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block;

[0116] Receive a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which a target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file;

[0117] The space block to be deleted is released according to the storage space release request.

[0118] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0119] Determine candidate space blocks whose space block states are unallocated from each storage space block;

[0120] Determine a target space block corresponding to the storage space allocation request from each candidate space block, and set the state of the target space block to the allocation request state;

[0121] Send the target spatial block to the client.

[0122] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0123] Change the state of the target space block from the allocation request state to the allocation confirmation state;

[0124] Obtain the reception confirmation message returned by the client based on the target space block;

[0125] The state of the target space block is changed from an allocation confirmed state to an allocated state based on receiving the confirmation message.

[0126] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0127] Determine a requested allocated space size corresponding to the storage space allocation request, and determine a target space size based on the requested allocated space size; wherein the target space size is greater than the requested allocated space size;

[0128] A target space block corresponding to the target space size is determined from each candidate space block.

[0129] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0130] Send the target space block to the client's space allocator;

[0131] Accordingly, the client uses the target space block to record the target file and the corresponding file header information, including:

[0132] The client obtains a target subspace block corresponding to the requested allocated space size from the target space block stored in its own space allocator, and uses the target subspace block to record the target file and the corresponding file header information.

[0133] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0134] The client determines the file header storage space from the target space block, generates the file header information of the target file including the size information of the target file, the file modification time, and the space mapping diagram, and saves the file header information to the file header storage space in the target space block.

[0135] In some specific implementations, the file header storage space includes a mapping storage space and an information storage space; the file header information is stored in the file header storage space in the target space block, and the processor can specifically implement the following steps by executing a computer program stored in the memory:

[0136] The space mapping map in the file header information is saved in the mapping map storage space;

[0137] The size information and file modification time in the file header information are saved to the information storage space.

[0138] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0139] Determine whether the size of the spatial map is greater than the size of the map storage space;

[0140] If the size of the spatial map is not larger than the size of the map storage space, the spatial map is saved in the map storage space;

[0141] If the size of the spatial mapping graph is larger than the size of the mapping graph storage space, the spatial mapping graph is split into multiple spatial mapping sub-graphs, and the target mapping sub-graphs in each spatial mapping sub-graph are saved in the mapping graph storage space based on a linked list mechanism.

[0142] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0143] The space block information of the space block to be deleted is stored in the space release mapping table according to the storage space release request; wherein the space block information of the space block to be deleted includes an identifier of the space block to be deleted, a size of the space block to be deleted, and a state of the space block to be deleted;

[0144] Changing the state of the to-be-deleted space block to a release request receiving state, generating a request receiving record, and persistently storing the request receiving record in a first preset disk;

[0145] Send a release confirmation request to the target non-central node and change the state of the space block to be deleted to the release pending confirmation state;

[0146] Receiving a release reconfirmation request returned by the target non-central node based on the release confirmation request;

[0147] Based on the release reconfirmation request, the space block to be deleted is released, and the state of the space block to be deleted is changed to unallocated state.

[0148] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0149] Based on the release confirmation request, the target non-central node changes the state of the space block to be deleted to the client deletion release record state, deletes the space block to be deleted, generates a space block deletion record, and persistently stores the space block deletion record in the second preset disk.

[0150] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0151] The space mapping diagram of the target file is read through the target non-central node to determine the space block to be deleted, the state of the space block to be deleted is set to a released state, and a space block to be deleted record is generated, and the space block to be deleted record is saved to a release tracking table in a third preset disk.

[0152] In some specific implementations, the processor can implement the following steps by executing the computer program stored in the memory:

[0153] The storage space release request and the space block to be deleted are sent to the central node through the target non-central node, and the status of the space block to be deleted is set to the release request status sent by the client.

[0154] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of this application, and it is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and it is not specifically limited here. In addition, the electronic device in this embodiment can be specifically an electronic computer.

[0155] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0156] The operating system 221 is used to manage and control various hardware devices on the electronic device and the computer program 222, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the storage space management method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0157] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the storage space management method disclosed above. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0158] Furthermore, an embodiment of the present application also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the storage space management method disclosed in any of the aforementioned embodiments.

[0159] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0160] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0161] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0162] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0163] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A storage space management method, characterized in that: The central node used in the distributed storage system includes: Get the storage space allocation request sent by the client; Sending the target space block corresponding to the storage space allocation request to the client, so that the client uses the target space block to record the target file and corresponding file header information; wherein the file header information includes a space mapping map for recording the mapping relationship between the target file and the target space block; Receive a storage space release request and a space block to be deleted sent by a target non-central node; wherein the target non-central node is a non-central node to which the target file belongs, and the space block to be deleted is a space block determined based on a space mapping diagram of the target file; Release the space of the to-be-deleted space block according to the storage space release request; Among them, the client uses the target space block to record the target file and the corresponding file header information, including: the client determines the file header storage space from the target space block, generates the file header information of the target file including the size information of the target file, the file modification time, and the space mapping map, and saves the file header information to the file header storage space in the target space block.

2. The storage space management method according to claim 1, characterized in that: The sending the target space block corresponding to the storage space allocation request to the client includes: Determine candidate space blocks whose space block states are unallocated from each storage space block; Determine a target space block corresponding to the storage space allocation request from each of the candidate space blocks, and set the state of the target space block to the allocation request state; The target spatial block is sent to the client.

3. The storage space management method according to claim 2, characterized in that: After sending the target spatial block to the client, the method further includes: Changing the state of the target space block from the allocation request state to an allocation confirmation state; Obtaining a reception confirmation message returned by the client based on the target space block; The state of the target space block is changed from the allocation confirmation state to an allocated state based on the reception confirmation message.

4. The storage space management method according to claim 3, characterized in that: The determining, from each of the candidate space blocks, a target space block corresponding to the storage space allocation request comprises: Determine a requested allocated space size corresponding to the storage space allocation request, and determine a target space size based on the requested allocated space size; wherein the target space size is larger than the requested allocated space size; A target space block corresponding to the target space size is determined from each of the candidate space blocks.

5. The storage space management method according to claim 4, characterized in that: The sending the target spatial block to the client includes: Sending the target space block to the space allocator of the client; Accordingly, the client uses the target space block to record the target file and the corresponding file header information, including: The client obtains a target subspace block corresponding to the requested allocated space size from the target space block stored in its own space allocator, and uses the target subspace block to record the target file and corresponding file header information.

6. The storage space management method according to claim 1, characterized in that: The file header storage space includes a mapping storage space and an information storage space; Accordingly, the step of saving the file header information to the file header storage space in the target space block includes: Saving the space mapping map in the file header information into the mapping map storage space; The size information and the file modification time in the file header information are saved in the information storage space.

7. The storage space management method according to claim 6, characterized in that: The step of saving the space mapping map in the file header information to the mapping map storage space includes: Determining whether the size of the spatial map is greater than the space size of the map storage space; If the size of the spatial map is not greater than the size of the map storage space, then storing the spatial map in the map storage space; If the size of the spatial mapping graph is larger than the space size of the mapping graph storage space, the spatial mapping graph is split into multiple spatial mapping sub-graphs, and the target mapping sub-graphs in each of the spatial mapping sub-graphs are saved in the mapping graph storage space based on a linked list mechanism.

8. The storage space management method according to any one of claims 1 to 7, characterized in that: The releasing the space of the to-be-deleted space block according to the storage space release request includes: The space block information of the space block to be deleted is stored in a space release mapping table according to the storage space release request; wherein the space block information of the space block to be deleted includes an identifier of the space block to be deleted, a size of the space block to be deleted, and a status of the space block to be deleted; Changing the state of the to-be-deleted space block to a release request receiving state, generating a request receiving record, and persistently storing the request receiving record in a first preset disk; Sending a release confirmation request to the target non-central node, and changing the state of the to-be-deleted space block to a release pending confirmation state; receiving a release reconfirmation request returned by the target non-central node based on the release confirmation request; Based on the release reconfirmation request, the space block to be deleted is released, and the state of the space block to be deleted is changed to an unallocated state.

9. The storage space management method according to claim 8, characterized in that: After sending the release confirmation request to the target non-central node, the method further includes: Based on the release confirmation request, the target non-central node changes the state of the space block to be deleted to the client deletion release record state, deletes the space block to be deleted, generates a space block deletion record, and persistently stores the space block deletion record in the second preset disk.

10. The storage space management method according to claim 8, characterized in that: Before receiving the storage space release request and the space block to be deleted sent by the target non-central node, the method further includes: The space mapping map of the target file is read through the target non-central node to determine the space block to be deleted, the state of the space block to be deleted is set to a released state, and a space block to be deleted record is generated, and the space block to be deleted record is saved to a release tracking table in a third preset disk.

11. The storage space management method according to claim 10, characterized in that: Also includes: The storage space release request and the space block to be deleted are sent to the central node through the target non-central node, and the state of the space block to be deleted is set to the release request state sent by the client.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the storage space management method described in any one of claims 1 to 11 are implemented.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the storage space management method as claimed in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the storage space management method according to any one of claims 1 to 11 are implemented.

Images