Storage space capacity detection method and device, computer equipment and storage medium

Abstract

The invention relates to a storage space capacity detection method and device, computer equipment and a storage medium. The method comprises the following steps: determining a target storage unit in a storage structure corresponding to a target storage space, analyzing key value data in the target storage unit to obtain analyzed data related to the target storage unit, and counting the analyzed data of all the target storage units to obtain capacity information of the target storage space; wherein the target storage unit is a storage unit with changed storage data. The capacity information of the target storage space is determined according to the key value data in the storage structure, the target storage space does not need to be traversed, the detection efficiency of the capacity of the storage space is improved, the capacity detection process is combined with the change process of the storage data, and compared with capacity statistics in the processing process of the read-write request in the prior art, the capacity detection efficiency is improved. And the read-write performance of the server is improved.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a method, device, computer equipment and storage medium for detecting storage space capacity. Background Art

[0002] All-flash storage array is a centralized storage system that uses Non-Volatile Memory Express (NVMe) hard disks as data storage media, with high throughput, low latency and high reliability, suitable for scenarios such as databases and high-performance computing. In an all-flash array system, the capacity of the storage space needs to be counted to manage the storage space.

[0003] In conventional technology, the capacity of the storage space is usually detected on an IO path when data is written into the storage space. However, the above storage space capacity detection method may affect the performance of the storage system. Summary of the invention

[0004] Based on this, it is necessary to provide a storage space capacity detection method, device, computer equipment and storage medium that can improve the performance of the storage system in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting storage space capacity, comprising:

[0006] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0007] Parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit;

[0008] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0009] In one of the embodiments, the parsed data includes: storage unit type, key value type, and data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type.

[0010] In one embodiment, statistics of parsed data of all target storage units are collected to obtain capacity information of the target storage space, including:

[0011] For each target storage unit, determining a capacity change direction of the target storage unit according to the storage unit type;

[0012] Determine the capacity change range of the target storage unit according to the key value type and data volume;

[0013] Determine the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit;

[0014] The capacity values ​​of all target storage units are counted to obtain the capacity information of the target storage space.

[0015] In one embodiment, determining the capacity change direction of the target storage unit according to the storage unit type includes:

[0016] If the storage unit type is an input type, the capacity change direction is determined to be a negative change direction;

[0017] If the storage unit type is an output type, the capacity change direction is determined to be a positive change direction.

[0018] In one embodiment, determining the capacity change range of the target storage unit according to the key value type and the data volume includes:

[0019] If the key value type is VBA type, the capacity change range is determined as the first change range according to the data volume;

[0020] If the storage unit type is UNMAP type, the capacity change range is determined as the second change range according to the data volume;

[0021] If the storage unit type is PATTERN type, the capacity change range is determined to be a third change range according to the data volume;

[0022] If the storage unit type is a compression type, the capacity change range is determined as a fourth change range according to the data amount.

[0023] In one embodiment, determining the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit includes:

[0024] If the capacity change direction of the target storage unit is negative, the capacity value of the capacity change amplitude is reduced;

[0025] If the capacity change direction of the target storage unit is positive, the capacity value of the capacity change amplitude is increased.

[0026] In one embodiment, the method further comprises:

[0027] When a merge instruction for merging storage units is received, a step of determining a target storage unit in a storage structure corresponding to a target storage space is executed, and a merge process is performed on the storage units indicated by the merge instruction.

[0028] In a second aspect, the present application further provides a storage space capacity detection device, comprising:

[0029] A first determining module is used to determine a target storage unit in a storage structure corresponding to a target storage space; the target storage unit is a storage unit in which storage data has changed;

[0030] A parsing module, used for parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit;

[0031] The statistics module is used to count the parsed data of all target storage units and obtain the capacity information of the target storage space.

[0032] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0034] Parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit;

[0035] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0038] Parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit;

[0039] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0040] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0041] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0042] Parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit;

[0043] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0044] The above-mentioned storage space capacity detection method, device, computer equipment and storage medium determine the target storage unit in the storage structure corresponding to the target storage space, and then parse the key value data in the target storage unit to obtain the parsed data related to the target storage unit, thereby counting the parsed data of all target storage units to obtain the capacity information of the target storage space; wherein the target storage unit is a storage unit whose storage data has changed. The capacity information of the target storage space is determined based on the key value data in the storage structure, without traversing the target storage space, thereby improving the detection efficiency of the storage space capacity, and combining the capacity detection process with the storage data change process, compared with the prior art of counting the capacity during the processing of read and write requests, the read and write performance of the server is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 A diagram of an application environment of a method for detecting storage space capacity in one embodiment;

[0047] Figure 2 is a schematic diagram of a flow chart of a method for detecting storage space capacity in one embodiment;

[0048] Figure 3 is a schematic diagram of a storage structure of a target storage space in one embodiment;

[0049] Figure 4 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0050] Figure 5 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0051] Figure 6 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0052] Figure 7 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0053] Figure 8 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0054] Fig. 9 is a schematic flow chart of a method for detecting storage space capacity in another embodiment;

[0055] Fig.10 It is a structural block diagram of a storage space capacity detection device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] The storage space capacity detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Wherein, the computer device can be a server, and the computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Wherein, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Wherein, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store detection data of storage space capacity. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting storage space capacity is implemented.

[0058] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0059] At present, the physical space composed of all non-volatile memory host controller interface specification (Non-Volatile Memory Express, NVMe) hard disks is virtually divided inside the all-flash array. The volume provided by the storage system to the user is a logical volume. The data read and write request on the volume corresponds to a set of parameters: volume identifier, offset address and read and write length. The read and write offset address will eventually form an address mapping relationship from LBA (Logic Block Address) to PBA. This mapping represents the conversion relationship between the address of the data on the logical volume and the physical address of the actual data storage. It is saved in the metadata management module as metadata. When the user reads the data of the volume, the metadata can provide address search and positioning. Metadata is crucial for data reading and writing. The organization form of metadata usually adopts the formation of key-value data (KV) for persistence, and provides hot spot acceleration such as memory cache for more efficient and fast access. Among them, KV data storage adopts the log-structured merge-tree (LSM) method. The LSM tree makes full use of the fact that the speed of disk batch sequential write is much higher than the random write speed. The data is first written into the static sorted table (SST) in memory, and then merged level by level using a multi-layer approach of memory and disk, which greatly improves the write performance.

[0060] In an all-flash array system, it is necessary to count the capacity of user volumes and storage pools. A common method is to count capacity changes on the IO path when data is written to the volume. For example, there is a write with parameters (1,100,8K), which means that in a volume with volume ID 1, an IO with a length of 8KB is written at the logical offset address 100. If the volume has not written data to this address before, the written capacity of the storage volume will increase by 8KB. If data has been written before, the old data will be overwritten and the capacity will not increase. Furthermore, the write bitmap of the storage volume can be searched with the help of metadata to determine whether this address has been written before in the logical address mapping table. For capacity statistics of the storage pool, it is necessary to count the used capacity of the storage pool based on the PBA occupancy status based on the physical address mapping table. The statistical method is the same as the search principle of the logical address mapping table. However, the above storage space capacity detection method will affect the performance of the storage system and the detection efficiency is low.

[0061] In view of this, an embodiment of the present application proposes a method for detecting storage space capacity, which can improve the security and performance of the trusted system of the trusted root system.

[0062] It should be noted that the beneficial effects brought about by the embodiments of the present application or the technical problems solved are not limited to this one, but may also include other implicit or related problems. For details, please refer to the description of the following embodiments.

[0063] In one embodiment, Figure 2 As shown, a method for detecting storage space capacity is provided, and the method is applied to Figure 1 The following is an example of a server in the example, including:

[0064] S201, determining a target storage unit in a storage structure corresponding to a target storage space; the target storage unit is a storage unit in which storage data has changed.

[0065] The target storage space can be a storage volume or a storage pool. Figure 3 As shown, the storage structure is a tree structure, and the storage structure includes multiple storage units. For example, the storage structure may be a log-structured merge-tree (LSM), and the storage unit may be a static sorted table (SST).

[0066] In the embodiment of the present application, the storage unit in the storage structure corresponding to the target storage space stores metadata, and the metadata records the write location and size of the user data, so that the capacity of the logical volume written data and the actual occupied capacity of the storage pool can be obtained with the help of the metadata statistical data. When the server obtains or initiates a modification instruction for the storage unit, the storage unit whose storage data has changed can be determined according to the modification instruction, and the storage unit whose storage data has changed is determined as the target storage unit in the storage structure corresponding to the target storage space.

[0067] S202, parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit.

[0068] Among them, Figure 3 As shown, the key-value data may be key-value pair data (Key-Value Data, KV). KV data is a method of storing and managing data in the form of key-value pairs. In an embodiment of the present application, metadata may be stored in a storage unit in the form of KV data.

[0069] In an embodiment of the present application, the key-value data in the target storage unit is parsed. For example, the data structure corresponding to the KV data, such as a hash table structure, can be determined first, and then the KV data can be read by separate linking method, open addressing and other methods to obtain parsed data related to the target storage unit.

[0070] Optionally, the parsed data may include identification information of the storage unit, order information of the storage unit, etc.

[0071] Optionally, in an embodiment of the present application, parsed data includes: storage unit type, key value type, data volume; storage unit type includes input type or output type; key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type. Among them, VBA is a general data mapping type, UNMAP type is usually related to memory management and virtual memory mapping, and PATTERN type is a special pattern type.

[0072] S203, counting the parsed data of all target storage units to obtain capacity information of the target storage space.

[0073] In the embodiment of the present application, the capacity value corresponding to each target storage unit can be determined according to the parsed data of each target storage unit, and further, the capacity values ​​corresponding to all target storage units can be summarized to obtain the capacity information of the target storage space. Optionally, the capacity change information of the target storage space can be determined according to the statistics of the parsed data of all target storage units, so as to obtain the capacity information of the target storage space according to the capacity change information and the original capacity information of the target storage space; or, the total occupied capacity information of all target storage units can be determined according to the parsed data of all target storage units, so as to obtain the capacity information of the target storage space according to the total occupied capacity information and the capacity information of other storage units in the storage structure.

[0074] Optionally, the target storage unit may include storage unit 1, storage unit 2 and storage unit 3. According to the parsed data of storage unit 1, the occupied capacity information corresponding to storage unit 1 is determined to be 3MB, the occupied capacity information corresponding to storage unit 2 is determined to be 5MB, and the occupied capacity information corresponding to storage unit 3 is determined to be 1MB. Then the total occupied capacity information of all target storage units is 9MB. If the capacity information of other storage units in the storage structure is 45MB, the capacity information of the target storage space is 54MB.

[0075] In the above storage space capacity detection method, the target storage unit in the storage structure corresponding to the target storage space is determined, and then the key value data in the target storage unit is parsed to obtain the parsed data related to the target storage unit, thereby counting the parsed data of all target storage units to obtain the capacity information of the target storage space; wherein the target storage unit is a storage unit whose storage data has changed. By determining the capacity information of the target storage space based on the key value data in the storage structure, there is no need to traverse the target storage space, which improves the detection efficiency of the storage space capacity, and combines the capacity detection process with the storage data change process, which improves the read and write performance of the server compared with the prior art of counting the capacity during the processing of read and write requests.

[0076] In one embodiment, an implementation of the above S203 is provided, such as Figure 4 As shown, the above “counting the parsed data of all target storage units to obtain the capacity information of the target storage space” includes:

[0077] S301 : for each target storage unit, determine a capacity change direction of the target storage unit according to the storage unit type.

[0078] The capacity change direction of the target storage unit includes a positive change direction or a negative change direction. The positive change direction indicates an increase in occupied capacity, and the negative change direction indicates a decrease in occupied capacity.

[0079] As an optional implementation, the storage unit type can be represented by a numerical value. When the storage unit type is 1, the capacity change direction of the target storage unit is determined to be a positive change direction; when the storage unit type is 2, the capacity change direction of the target storage unit is determined to be a negative change direction.

[0080] As another optional implementation, the storage unit type can be represented by characters. When the storage unit type is a character representing growth, the capacity change direction of the target storage unit is determined to be a positive change direction; when the storage unit type is a character representing decrease, the capacity change direction of the target storage unit is determined to be a negative change direction.

[0081] S302: Determine the capacity change range of the target storage unit according to the key value type and the data volume.

[0082] In the embodiment of the present application, the data volume corresponding to each key-value data in the target storage unit may be the same or different. It is possible to determine whether the capacity change range of the target storage unit is the data volume according to the key-value type. Optionally, the data volume corresponding to the key-value data may be 8KB, and the key-value type may be divided into a first type and a second type. When the key-value type of the key-value data is the first type, the capacity change range of the key-value data is determined to be the data volume; when the key-value type corresponding to the key-value data is the second type, the capacity change range of the key-value data is determined according to a preset determination logic or input information.

[0083] In an embodiment of the present application, each target storage unit includes at least one key-value data. The capacity change range of the target storage unit is determined based on the capacity change ranges corresponding to all the key-value data. For example, the target storage unit includes key-value data 1, key-value data 2, and key-value data 3. The capacity change range of key-value data 1 is 8KB, the capacity change range of key-value data 2 is 16KB, and the capacity change range of key-value data 3 is 4KB. Then, the capacity change range of the target storage unit is 28KB.

[0084] S303: Determine the capacity value of the target storage unit according to the capacity change direction and capacity change range of the target storage unit.

[0085] In the embodiment of the present application, the capacity value of the target storage unit may be a capacity change value of the target storage unit or an occupied capacity value of the target storage unit. Optionally, the capacity change value of the target storage unit may be determined based on the capacity change direction and capacity change amplitude of the target storage unit; or, the occupied capacity value of the target storage unit may be determined based on the capacity change direction and capacity change amplitude of the target storage unit, and a preset occupied capacity value.

[0086] As an optional implementation, the capacity value of the target storage unit can be determined according to the capacity change direction, capacity change range and preset ratio value of the target storage unit. For example, if the capacity change direction of the target storage unit is a positive change direction, the capacity change range is 8KB, and the preset ratio value is 50%, the capacity value of the target storage unit is determined to be 16KB; if the capacity change direction of the target storage unit is a negative change direction, the capacity change range is 8KB, and the preset ratio value is 50%, the capacity value of the target storage unit is determined to be -16KB.

[0087] As another optional implementation, Figure 5 As shown, the above “determining the capacity value of the target storage unit according to the capacity change direction and capacity change range of the target storage unit” includes:

[0088] S3031: If the capacity change direction of the target storage unit is negative, reduce the capacity value of the capacity change amplitude.

[0089] S3032: If the capacity change direction of the target storage unit is positive, increase the capacity value of the capacity change amplitude.

[0090] Optionally, if the capacity change direction of the target storage unit is positive and the capacity change amplitude is 8KB, the capacity value of the target storage unit is determined to be 8KB; if the capacity change direction of the target storage unit is negative and the capacity change amplitude is 8KB, the capacity value of the target storage unit is determined to be -8KB.

[0091] S304, counting the capacity values ​​of all target storage units to obtain the capacity information of the target storage space.

[0092] In the embodiment of the present application, the capacity values ​​of all target storage units are summarized to obtain the capacity information of the target storage space. Optionally, the capacity change information of the target storage space can be determined based on the statistical analysis data of all target storage units, thereby obtaining the capacity information of the target storage space based on the capacity change information and the original capacity information of the target storage space; or, the total occupied capacity information of all target storage units can be determined based on the analysis data of all target storage units, thereby obtaining the capacity information of the target storage space based on the total occupied capacity information and the capacity information of other storage units in the storage structure.

[0093] In the above application embodiment, the capacity value of each target storage unit is determined based on the storage unit type, key value type, and data volume corresponding to each key value data, so that the parsed data of all target storage units are counted to obtain the capacity information of the target storage space, thereby improving the accuracy of the capacity information of the target storage space.

[0094] In one embodiment, an implementation of the above S301 is provided, such as Figure 6 As shown, the above “determining the capacity change direction of the target storage unit according to the storage unit type” includes:

[0095] S3011: If the storage unit type is an input type, determine that the capacity change direction is a negative change direction.

[0096] In an embodiment of the present application, if the storage unit type is an input type, indicating that the target storage unit is a storage unit before the merge, or the data in the target storage unit is newly written data, then the capacity change information corresponding to the target storage unit is a decrease in available storage capacity, that is, the capacity change direction is determined to be a negative change direction.

[0097] S3012: If the storage unit type is an output type, determine that the capacity change direction is a positive change direction.

[0098] In an embodiment of the present application, if the storage unit type is an output type, indicating that the target storage unit is a merged storage unit, or the data in the target storage unit is newly written data, then the capacity change information corresponding to the target storage unit is an increase in available storage capacity, that is, the capacity change direction is determined to be a positive change direction.

[0099] In the above-mentioned application embodiment, the capacity change direction of the target storage unit is quickly determined according to the storage unit type. When the storage unit type is an input type, the capacity change direction is determined to be a negative change direction, and when the storage unit type is an output type, the capacity change direction is determined to be a positive change direction. This improves the efficiency of determining the capacity change direction of the target storage unit, thereby improving the efficiency of detecting the storage space capacity.

[0100] In one embodiment, an implementation of the above S302 is provided, such as Figure 7 As shown, the above “determining the capacity change range of the target storage unit according to the key value type and the data volume” includes:

[0101] S3021: If the key value type is a VBA type, the capacity change range is determined as a first change range according to the data volume.

[0102] In an embodiment of the present application, the VBA type is a general data mapping type, which represents the mapping relationship between the logical addressing (Logical Block Addressing, LBA) and the physical block address PBA, wherein LBA is a hard disk data storage and access method, and PBA is an address system used in a solid-state hard disk to describe the specific storage location of data on the flash memory. If the key value type is a VBA type, then the data that represents the actual corresponding data of the key value data is the same as the data amount, and the capacity change amplitude corresponding to the key value data is determined to be the data amount. Furthermore, if the target storage unit includes one key value data, the data amount is used as the first change amplitude; if the target storage unit includes multiple key value data, the sum of the capacity change amplitudes of the multiple key value data is used as the first change amplitude.

[0103] S3022: If the key value type is UNMAP type, determine the capacity change range as the second change range according to the data volume.

[0104] In the embodiment of the present application, the UNMAP type is generally related to memory management and virtual memory mapping. If the key value type is the UNMAP type, the data actually corresponding to the key value data may be 0, and the capacity change range corresponding to the key value data is determined as the product of the data volume and the first preset value, and the first preset value may be 0. Further, if the target storage unit includes one key value data, the product of the data volume and the first preset value is used as the second change range; if the target storage unit includes multiple key value data, the sum of the capacity change ranges of the multiple key value data is used as the second change range.

[0105] S3023: If the key value type is PATTERN type, the capacity change range is determined to be a third change range based on the data volume.

[0106] In this embodiment, the PATTERN type is a special storage type, such as a data structure for storing a regular expression pattern. If the key value type is the PATTERN type, the data actually corresponding to the key value data is the same as the data volume, and the capacity change range corresponding to the key value data is determined to be the data volume. Further, if the target storage unit includes one key value data, the data volume is used as the third change range; if the target storage unit includes multiple key value data, the sum of the capacity change ranges of the multiple key value data is used as the third change range.

[0107] S3024: If the key value type is a compression type, the capacity change range is determined to be a fourth change range based on the data volume.

[0108] In the embodiment of the present application, if the key value type is a compression type, the capacity change range corresponding to the key value data is determined according to the product of the data volume and the second preset value, wherein the second preset value may be a compression ratio corresponding to the compression type. Further, if the target storage unit includes one key value data, the product of the data volume and the second preset value is used as the fourth change range; if the target storage unit includes multiple key value data, the sum of the capacity change ranges of the multiple key value data is used as the fourth change range.

[0109] In the above application embodiments, when the key value type is VBA type, UNMAP type, PATTERN type and compression type, the capacity change amplitude corresponding to each key value data is determined, thereby obtaining the capacity change amplitude of the target storage unit, thereby improving the accuracy of the capacity change amplitude.

[0110] In one embodiment, Figure 8 As shown, the above storage space capacity detection method also includes:

[0111] S204, when a merge instruction for merging storage units is received, a step of determining a target storage unit in a storage structure corresponding to the target storage space is executed, and a merge process is performed on the storage units indicated by the merge instruction.

[0112] In an embodiment of the present application, the storage structure corresponding to the target storage space may be an LSM tree, and the storage unit in the LSM tree may be an SST file. Optionally, the modification instruction for the storage unit may be an internal merge instruction of the LSM tree corresponding to the target storage space. The server will aggregate to form a new SST file according to the internal merge instruction and delete the corresponding old SST file, thereby determining the changed SST file as the target storage unit.

[0113] Optionally, merge instructions may be generated based on storage addresses and for duplicates and old data.

[0114] In the above application embodiment, according to the merge instruction of merging the storage units, the storage units indicated by the merge instruction are merged to realize the step-by-step merging and sinking in the storage architecture, and the key value data of the same storage address is processed so that duplicate data and old data are processed.

[0115] In one embodiment, a complete storage space capacity detection method is also provided, such as Fig. 9 As shown, including:

[0116] S1, when receiving a merge instruction for merging storage units, execute the step of determining the target storage unit in the storage structure corresponding to the target storage space, and merge the storage units indicated by the merge instruction; the target storage unit is the storage unit whose storage data has changed.

[0117] S2, parse the key value data in the target storage unit to obtain parsed data related to the target storage unit; the parsed data includes: storage unit type, key value type, data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type.

[0118] S3: For each target storage unit, execute S4 or S5.

[0119] S4: If the storage unit type is an input type, determine that the capacity change direction is a negative change direction.

[0120] S5: If the storage unit type is an output type, determine that the capacity change direction is a positive change direction.

[0121] S6: If the key value type is a VBA type, the capacity change range is determined as a first change range according to the data volume.

[0122] S7: If the key value type is UNMAP type, determine the capacity change range as the second change range according to the data volume.

[0123] S8. If the key value type is PATTERN type, the capacity change range is determined to be a third change range according to the data volume.

[0124] S9: If the key value type is a compression type, the capacity change range is determined as a fourth change range according to the data volume.

[0125] S10: If the capacity change direction of the target storage unit is negative, then reduce the capacity value of the capacity change amplitude.

[0126] S11, if the capacity change direction of the target storage unit is a positive direction, then increase the capacity value of the capacity change amplitude.

[0127] S12, counting the capacity values ​​of all target storage units to obtain the capacity information of the target storage space.

[0128] In the above storage space capacity detection method, the target storage unit in the storage structure corresponding to the target storage space is determined, and then the key value data in the target storage unit is parsed to obtain the parsed data related to the target storage unit, thereby counting the parsed data of all target storage units to obtain the capacity information of the target storage space; wherein the target storage unit is a storage unit whose storage data has changed. By determining the capacity information of the target storage space based on the key value data in the storage structure, there is no need to traverse the target storage space, which improves the detection efficiency of the storage space capacity, and combines the capacity detection process with the storage data change process, which improves the read and write performance of the server compared with the prior art of counting the capacity during the processing of read and write requests.

[0129] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0130] Based on the same inventive concept, the embodiment of the present application also provides a storage space capacity detection device for implementing the storage space capacity detection method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more storage space capacity detection devices provided below can refer to the limitations of the storage space capacity detection method above, and will not be repeated here.

[0131] In one embodiment, Fig.10 As shown, a storage space capacity detection device is provided, comprising: a determination module 10, an analysis module 11 and a statistical module 12, wherein:

[0132] The determination module 10 is used to determine a target storage unit in a storage structure corresponding to a target storage space; the target storage unit is a storage unit in which storage data has changed.

[0133] The parsing module 11 is used to parse the key value data in the target storage unit to obtain parsed data related to the target storage unit; the parsed data includes: storage unit type, key value type, data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type.

[0134] The statistics module 12 is used to count the parsed data of all target storage units to obtain the capacity information of the target storage space.

[0135] In one embodiment, the statistical module 12 includes: a first determining unit, a second determining unit, a third determining unit and a statistical unit, wherein:

[0136] The first determining unit is used to determine, for each target storage unit, a capacity change direction of the target storage unit according to the storage unit type.

[0137] The second determining unit is used to determine the capacity change range of the target storage unit according to the key value type and the data volume.

[0138] The third determining unit is used to determine the capacity value of the target storage unit according to the capacity change direction and capacity change range of the target storage unit.

[0139] The statistical unit is used to count the capacity values ​​of all target storage units to obtain the capacity information of the target storage space.

[0140] In one embodiment, the first determination unit is specifically used to determine that the capacity change direction is a negative change direction when the storage unit type is an input type; and to determine that the capacity change direction is a positive change direction when the storage unit type is an output type.

[0141] In one embodiment, the above-mentioned second determination unit is specifically used to determine that the capacity change amplitude is a first change amplitude according to the data volume when the key value type is a VBA type; when the key value type is an UNMAP type, determine that the capacity change amplitude is a second change amplitude according to the data volume; when the key value type is a PATTERN type, determine that the capacity change amplitude is a third change amplitude according to the data volume; when the key value type is a compression type, determine that the capacity change amplitude is a fourth change amplitude according to the data volume.

[0142] In one embodiment, the third determination unit is specifically used to reduce the capacity value of the capacity change amplitude when the capacity change direction of the target storage unit is negative; and to increase the capacity value of the capacity change amplitude when the capacity change direction of the target storage unit is positive.

[0143] In one embodiment, the above storage space capacity detection device further includes: a merging module.

[0144] The determination module is specifically used to execute the step of determining the target storage unit in the storage structure corresponding to the target storage space when receiving a merging instruction for merging storage units.

[0145] The merging module is used to merge the storage units indicated by the merging instruction when receiving the merging instruction of merging the storage units.

[0146] Each module in the above storage space capacity detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0147] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0148] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0149] Parse the key value data in the target storage unit to obtain parsed data related to the target storage unit; the parsed data includes: storage unit type, key value type, data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type;

[0150] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0151] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0152] For each target storage unit, determining a capacity change direction of the target storage unit according to the storage unit type;

[0153] Determine the capacity change range of the target storage unit according to the key value type and data volume;

[0154] Determine the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit;

[0155] The capacity values ​​of all target storage units are counted to obtain the capacity information of the target storage space.

[0156] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0157] If the storage unit type is an input type, the capacity change direction is determined to be a negative change direction;

[0158] If the storage unit type is an output type, the capacity change direction is determined to be a positive change direction.

[0159] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0160] If the key value type is VBA type, the capacity change range is determined as the first change range according to the data volume;

[0161] If the key value type is UNMAP type, the capacity change range is determined as the second change range according to the data volume;

[0162] If the key value type is PATTERN type, the capacity change range is determined to be the third change range based on the data volume;

[0163] If the key value type is a compression type, the capacity change range is determined as the fourth change range according to the data volume.

[0164] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0165] If the capacity change direction of the target storage unit is negative, the capacity value of the capacity change amplitude is reduced;

[0166] If the capacity change direction of the target storage unit is positive, the capacity value of the capacity change amplitude is increased.

[0167] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0168] When a merge instruction for merging storage units is received, a step of determining a target storage unit in a storage structure corresponding to a target storage space is executed, and a merge process is performed on the storage units indicated by the merge instruction.

[0169] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0170] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0171] Parse the key value data in the target storage unit to obtain parsed data related to the target storage unit; the parsed data includes: storage unit type, key value type, data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type;

[0172] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0174] For each target storage unit, determining a capacity change direction of the target storage unit according to the storage unit type;

[0175] Determine the capacity change range of the target storage unit according to the key value type and data volume;

[0176] Determine the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit;

[0177] The capacity values ​​of all target storage units are counted to obtain the capacity information of the target storage space.

[0178] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0179] If the storage unit type is an input type, the capacity change direction is determined to be a negative change direction;

[0180] If the storage unit type is an output type, the capacity change direction is determined to be a positive change direction.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] If the key value type is VBA type, the capacity change range is determined as the first change range according to the data volume;

[0183] If the key value type is UNMAP type, the capacity change range is determined as the second change range according to the data volume;

[0184] If the key value type is PATTERN type, the capacity change range is determined to be the third change range based on the data volume;

[0185] If the key value type is a compression type, the capacity change range is determined as the fourth change range according to the data volume.

[0186] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0187] If the capacity change direction of the target storage unit is negative, the capacity value of the capacity change amplitude is reduced;

[0188] If the capacity change direction of the target storage unit is positive, the capacity value of the capacity change amplitude is increased.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0190] When a merge instruction for merging storage units is received, a step of determining a target storage unit in a storage structure corresponding to a target storage space is executed, and a merge process is performed on the storage units indicated by the merge instruction.

[0191] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0192] Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which storage data has changed;

[0193] Parse the key value data in the target storage unit to obtain parsed data related to the target storage unit; storage unit type, key value type, data volume; storage unit type includes input type or output type; key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type;

[0194] The parsed data of all target storage units are counted to obtain the capacity information of the target storage space.

[0195] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0196] For each target storage unit, determining a capacity change direction of the target storage unit according to the storage unit type;

[0197] Determine the capacity change range of the target storage unit according to the key value type and data volume;

[0198] Determine the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit;

[0199] The capacity values ​​of all target storage units are counted to obtain the capacity information of the target storage space.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] If the storage unit type is an input type, the capacity change direction is determined to be a negative change direction;

[0202] If the storage unit type is an output type, the capacity change direction is determined to be a positive change direction.

[0203] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0204] If the key value type is VBA type, the capacity change range is determined as the first change range according to the data volume;

[0205] If the key value type is UNMAP type, the capacity change range is determined as the second change range according to the data volume;

[0206] If the key value type is PATTERN type, the capacity change range is determined to be the third change range based on the data volume;

[0207] If the key value type is a compression type, the capacity change range is determined as the fourth change range according to the data volume.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0209] If the capacity change direction of the target storage unit is negative, the capacity value of the capacity change amplitude is reduced;

[0210] If the capacity change direction of the target storage unit is positive, the capacity value of the capacity change amplitude is increased.

[0211] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0212] When a merge instruction for merging storage units is received, a step of determining a target storage unit in a storage structure corresponding to a target storage space is executed, and a merge process is performed on the storage units indicated by the merge instruction.

[0213] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0214] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0215] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting storage space capacity, characterized in that: The method comprises: Determine a target storage unit in a storage structure corresponding to the target storage space; the target storage unit is a storage unit in which the storage data has changed; Parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit; The parsed data of all the target storage units are counted to obtain the capacity information of the target storage space.

2. The method according to claim 1, characterized in that: The parsed data includes: storage unit type, key value type, data volume; the storage unit type includes input type or output type; the key value type includes any one of VBA type, UNMAP type, PATTERN type, and compression type.

3. The method according to claim 2, characterized in that The counting of the parsed data of all the target storage units to obtain the capacity information of the target storage space includes: For each of the target storage units, determining a capacity change direction of the target storage unit according to the storage unit type; Determining a capacity change range of the target storage unit according to the key value type and the data volume; Determining a capacity value of the target storage unit according to a capacity change direction and a capacity change amplitude of the target storage unit; The capacity values ​​of all the target storage units are counted to obtain the capacity information of the target storage space.

4. The method according to claim 3, characterized in that The determining the capacity change direction of the target storage unit according to the storage unit type includes: If the storage unit type is the input type, determining that the capacity change direction is a negative change direction; If the storage unit type is the output type, it is determined that the capacity change direction is a positive change direction.

5. The method according to claim 3, characterized in that: The determining, according to the key value type and the data volume, the capacity change range of the target storage unit includes: If the key value type is the VBA type, determining the capacity change range as a first change range according to the data volume; If the key value type is the UNMAP type, determining the capacity change range as a second change range according to the data volume; If the key value type is the PATTERN type, determining the capacity change range as a third change range according to the data volume; If the key value type is the compression type, the capacity change range is determined to be a fourth change range according to the data volume.

6. The method according to claim 3, characterized in that The determining the capacity value of the target storage unit according to the capacity change direction and capacity change amplitude of the target storage unit includes: If the capacity change direction of the target storage unit is negative, reducing the capacity value of the capacity change amplitude; If the capacity change direction of the target storage unit is a positive direction, the capacity value of the capacity change amplitude is increased.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When a merge instruction for merging storage units is received, the step of determining the target storage unit in the storage structure corresponding to the target storage space is executed, and the storage units indicated by the merge instruction are merged.

8. A device for detecting storage space capacity, characterized in that: The device comprises: A determination module, used to determine a target storage unit in a storage structure corresponding to a target storage space; the target storage unit is a storage unit in which storage data has changed; A parsing module, used for parsing the key-value data in the target storage unit to obtain parsed data related to the target storage unit; The statistics module is used to count the parsed data of all the target storage units to obtain the capacity information of the target storage space.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images