Storage capacity determination method based on hard disk, storage system, server and medium

Abstract

The invention discloses a storage capacity determination method based on a hard disk, a storage system, a server and a medium, and relates to the technical field of computers. When the current use capacity of the hard disk is greater than a capacity threshold value, carrying out current limiting processing on data issued to the cache by the host, so as to prolong response time of writing IO by the host by reducing the data volume subjected to the current limiting processing, and providing time for counting the current storage data by the cache, so that the cache is conveniently refreshed to the hard disk, and the phenomenon that when the volume of the host is switched to a read-only mode, the cache cannot be read is avoided. And the data in the cache corresponding to the left host fails to be written, so that the security of the data is improved. And brushing the cached current storage data down to the hard disk, counting the final storage data actually falling into the hard disk to serve as the updated current use capacity of the hard disk, and finally determining the mounting mode of the volume. The problem that the user experience is reduced due to the fact that the difference between storage data visible by a user and the actual storage capacity of a hard disk is large is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method for determining storage capacity based on a hard disk, a storage system, a server and a medium. Background Art

[0002] In the storage system, the host first writes data to the memory cache, and then the data in the memory cache is written to the hard disk in a flushing manner.

[0003] When the storage data in the hard disk reaches a certain threshold, the volume will switch to read-only mode. However, the comparison between the storage data in the hard disk and the threshold corresponds to the storage data visible to the user. Since the data in the memory cache includes both write data and delete data, it is possible that the final storage data will be less than the threshold when it actually falls into the hard disk, but the volume has been switched to read-only mode at this time, resulting in a large difference between the storage data visible to the user and the actual storage capacity of the hard disk. At the same time, when the volume switches to read-only mode, the data written to the memory cache by the host may fail to be written during the writing process, resulting in reduced data security and reduced user experience.

[0004] Therefore, how to improve the user experience and the security of data writing is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a method for determining storage capacity based on a hard disk, a storage system, a server and a medium to solve the problems of reduced user experience and reduced data security.

[0006] In order to solve the above technical problems, the present invention provides a method for determining storage capacity based on a hard disk, comprising:

[0007] Get the current used capacity of the hard disk and the capacity threshold visible to the outside world;

[0008] When the currently used capacity is greater than the capacity threshold, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data and then sent to the cache;

[0009] The cached current storage data is flushed to the hard disk, and the current used capacity of the hard disk is updated, so as to determine the volume mounting mode according to the updated current used capacity of the hard disk and the capacity threshold.

[0010] On the one hand, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including:

[0011] Acquire the critical usage capacity of the hard disk and the rate at which the host sends current data to the cache; wherein the critical usage capacity is greater than the capacity threshold;

[0012] Determine a first target current limiting level according to the current usage capacity and the critical usage capacity;

[0013] The rate of the current data is adjusted according to the current limiting rate corresponding to the first target current limiting level to complete the current limiting processing sent by the host to the cache.

[0014] On the other hand, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including:

[0015] Obtaining the unit request quantity and the threshold request quantity corresponding to the cache sent by the host; wherein the unit request quantity is greater than the threshold request quantity;

[0016] Determine the number of current limiting requests according to the number of unit requests and the number of threshold requests;

[0017] The data sent by the host to the cache is sent according to the number of current limiting requests to complete the current limiting processing sent by the host to the cache.

[0018] On the other hand, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including:

[0019] Obtaining an initial rate and a critical current limiting rate of current data sent by the host to the cache;

[0020] The initial rate of the current data is adjusted according to the critical current limiting rate to complete the current limiting processing sent by the host to the cache.

[0021] On the other hand, flushing the cached current storage data to the hard disk includes:

[0022] Obtain the cache reserved space of the hard disk and the write speed of the hard disk; wherein the storage data capacity of the cache reserved space is greater than the storage data capacity of the cache; and the rate corresponding to the write speed of the hard disk is greater than the corresponding current limiting rate after the host sends it to the cache;

[0023] receiving the currently stored data of the cache;

[0024] Determining a refresh time of the hard disk according to the current storage data and the write speed;

[0025] The currently stored data is flushed to the cache reserved space of the hard disk according to the flush time.

[0026] On the other hand, updating the currently used capacity of the hard disk includes:

[0027] Obtaining data capacities corresponding to write data and delete data of the currently stored data in the cache;

[0028] Adding the current used capacity of the hard disk to the data capacity of the written data to obtain a first data capacity;

[0029] The first data capacity is subtracted from the data capacity of the deleted data to obtain the updated current usage capacity of the hard disk.

[0030] On the other hand, determining the volume mount mode according to the updated current used capacity of the hard disk and the capacity threshold includes:

[0031] If the current used capacity of the hard disk after update is greater than or equal to the capacity threshold, determining that the mount mode of the volume is read-only mode;

[0032] If the current used capacity of the hard disk after update is less than the capacity threshold, determining that the mount mode of the volume is a read-write mode;

[0033] Correspondingly, after determining the volume mount mode, it also includes:

[0034] When the volume is mounted in read-only mode, switch the read-write business mode to query mode;

[0035] When the mount mode of the volume is the read-write mode, subtracting the updated current used capacity of the hard disk from the capacity threshold to obtain a second data capacity;

[0036] Pre-acquire a mapping relationship between a data capacity range and a current limiting level; wherein the data capacity range and the current limiting level are in a one-to-one mapping relationship;

[0037] Determining a target data capacity range corresponding to the second data capacity in the mapping relationship;

[0038] Determine a corresponding second target current limiting level according to the target data capacity range;

[0039] The limited data is sent to the cache according to the limited rate corresponding to the second target limited level.

[0040] To solve the above technical problems, the present invention further provides a storage system, comprising a host and a hard disk; wherein the cache is located on the host;

[0041] The host is used to execute the steps of the above-mentioned method for determining storage capacity based on a hard disk.

[0042] In order to solve the above technical problems, the present invention further provides a server, comprising:

[0043] Memory for storing computer programs;

[0044] A processor is used to implement the steps of the hard disk-based storage capacity determination method when executing the computer program.

[0045] In order to solve the above technical problem, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the hard disk-based storage capacity determination method are implemented.

[0046] The beneficial effect of the present invention is that, on the one hand, when the current use capacity of the hard disk is greater than the capacity threshold, the present invention will not immediately switch the mount mode of the volume, but will perform current limiting processing on the data sent to the cache by the corresponding host, so as to extend the response time of the host writing IO by reducing the amount of data processed by current limiting. At the same time, before the mount mode of the volume is changed to read-only mode, the data written to the cache by the host is delayed and reduced by current limiting processing, and the time for the cache to count the current stored data is given to facilitate the flushing to the hard disk, so as to avoid the failure of writing the data in the corresponding cache of the host left when the volume is switched to read-only mode, thereby improving the security of the data. On the other hand, the current stored data of the cache is flushed to the hard disk, and considering that the stored data of the cache includes write data and delete data, the final stored data actually falling into the hard disk is counted as the current use capacity of the hard disk after the update, and the mount mode of the volume is finally determined according to the current use capacity of the hard disk after the update and the capacity threshold. This is to avoid the problem of reduced user experience caused by the large difference between the storage data visible to the user and the storage capacity of the actual hard disk. The present invention fully considers the possibility that when the deleted data in the memory cache is greater than the written data, the current usage capacity of the corresponding updated hard disk will be smaller than the current usage capacity of the hard disk before the update, and sets the mounting mode of the volume to read-only mode prematurely, thereby improving the user experience.

[0047] Secondly, the first target current limiting level is determined based on the current usage capacity and the critical usage capacity, and the current data rate is adjusted to complete the current limiting processing with the current limiting rate corresponding to the first target current limiting level. Current limiting is achieved through rate adjustment of data traffic, extending the response time of host write IO, delaying and reducing the data written to the cache by the host, and simplifying the current limiting operation. The current limiting of the data sent to the cache by the host is affected by the reduction in the number of data requests, and the current limiting mechanism is dynamically adjusted according to the number of requests to achieve data current limiting processing, protect the backend data from being affected by too many requests, and ensure the stability and responsiveness of the system. By adjusting the initial speed to the critical current limiting rate in one step to achieve the highest level of current limiting, the extension time of the host write IO response time is maximized. Based on the current storage data of the cache, the current usage capacity of the hard disk is updated. Considering that the current storage data of the cache is flushed to the hard disk, the final storage data actually falling into the hard disk is counted as the current usage capacity of the updated hard disk, so as to realize the final determination of the volume mounting mode according to the current usage capacity and capacity threshold of the updated hard disk, so that it can reach the real volume mounting mode. It is determined by fully considering the original usage capacity of the hard disk and the cached storage data. The flush time is determined by the current write speed, and the amount of write cache written is controlled according to the flush time to ensure that the capacity of the flush cache does not exceed the cache reserved space of the hard disk, protect the data from being overwritten, and improve data security. The process of determining the volume mount mode, and in the corresponding read-only mode, the host's read and write business is changed to a query business, and in the corresponding read-write mode, preventive current limiting processing is performed to improve the user experience.

[0048] In addition, the present invention also provides a storage system, a server and a medium, which have the same beneficial effects as the above-mentioned hard disk-based storage capacity determination method. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A flowchart of a method for determining storage capacity based on a hard disk provided by an embodiment of the present invention;

[0051] Figure 2 A schematic diagram of a fast disk flushing of cached data provided by an embodiment of the present invention;

[0052] Figure 3 A structural diagram of a storage capacity determination device based on a hard disk provided in an embodiment of the present invention;

[0053] Figure 4 A structural diagram of a server provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The core of the present invention is to provide a method for determining storage capacity based on a hard disk, a storage system, a server and a medium to solve the problems of reduced user experience and reduced data security.

[0056] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0057] In the storage industry, especially in all-flash storage systems, the wear leveling of solid-state drives (SSDs) is generally achieved by appending writes. When counting capacity, the capacity will vary within a range. This range is the amount of cached data in the cache module. The corresponding data in the cache may be an overwrite of previous data or a write of new data. These amounts cannot be predicted in advance. An L1P1 insertion usually refers to the data cache in the L1 cache (Level 1 Cache). The L1 cache is a part of the central processing unit (CPU) that is used to store recently accessed data for quick access. The L1 cache is usually divided into the instruction cache and the data cache, where L1P1 refers to the first level of the L1 data cache. There may be two actions I (L1P1) and I (P1L1), or it may be split into three actions: D (P0L0), I (L1P1), and I (P1L1). For the operation split into two actions, the corresponding pool capacity statistics only increase. For the operation split into three actions, the three actions are executed randomly when writing and flushing the disk. It is possible that I (L1P1) and I (P1L1) are executed first, and then D (P0L0); it is also possible that D (P0L0) is executed first, and then I (L1P1) and I (P1L1). If D (P0L0) is executed first, the capacity statistics are relatively accurate. If I (L1P1) and I (P1L1) are executed first, and D (P0L0) is executed later, the capacity statistics will increase. When the deleted data is greater than the written data, the capacity statistics at a certain moment are inaccurate. The full flash pool will reserve a certain capacity to undertake the cached data. When the capacity reaches the read-only condition of the capacity, the volume will become read-only. In fact, at this time, it is possible that the volume does not meet the read-only condition, but provides the user with a read-only volume, which will give the user a bad experience and bring unnecessary trouble. The read-only condition is not met, but the volume on the storage pool is already read-only. At the host level, the writing process sent will fail. The storage capacity determination method based on the hard disk provided by the present invention can solve the above technical problems.

[0058] Figure 1 A flowchart of a method for determining storage capacity based on a hard disk is provided in an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0059] S11: obtaining the current used capacity of the hard disk and the capacity threshold visible to the outside;

[0060] S12: When the current used capacity is greater than the capacity threshold, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data and then sent to the cache;

[0061] S13: flushing the cached current storage data to the hard disk, and updating the current used capacity of the hard disk, so as to determine the volume mounting mode according to the updated current used capacity of the hard disk and the capacity threshold.

[0062] Specifically, the current used capacity of the hard disk corresponds to the capacity storage area visible to the user in the hard disk, and the externally visible capacity threshold is a preset user-visible threshold. In the conventional scheme, when the current used capacity is greater than the capacity threshold, it is necessary to modify the mount mode of the volume corresponding to the hard disk to read-only mode, so as to facilitate the host's business to perform query operations. It should be noted that this embodiment takes into account that the data flushed to the hard disk by the corresponding cache in the hard disk is not counted. There may be two situations where the cached write data is greater than or equal to the deleted data, and there may be two situations where the deleted data is greater than the written data. If the deleted data is greater than the written data, when it is flushed to the hard disk, the corresponding current used capacity will be less than the used capacity in step S11. After this happens, due to the premature modification of the volume mount mode to read-only, its actual used capacity does not reach the capacity threshold, reducing the user experience.

[0063] In step S12, when the current usage capacity is greater than the capacity threshold, the volume mount mode is not modified first, but the data sent from the host to the cache is subjected to flow limiting processing to obtain the data after flow limiting, and then sent to the cache. It should be noted that in this embodiment, the flow limiting processing of the data sent from the host to the cache is to extend the response time of the host write IO, delay and reduce the data written by the host to the cache, and provide enough time for the cache to count the current stored data and flush it to the hard disk. The flow limiting process here can limit the data flow, or limit the corresponding number of requests, which is not limited here. Corresponding to the data flow limiting process, it can be limited to the highest level (minimum data flow) in one step, and the corresponding flow limiting level can also be determined according to the gap between the current usage capacity and the critical capacity. The flow limiting levels here are divided into different levels, and the size of the data flow limit at different levels is different, so as to ensure that the real-time usage capacity of the hard disk is counted when the speed of the host sending data is appropriate.

[0064] In some embodiments, performing flow limiting processing on data sent from the host to the cache to obtain flow-limited data includes:

[0065] Obtaining the critical usage capacity of the hard disk and the rate at which the host sends current data to the cache; wherein the critical usage capacity is greater than the capacity threshold;

[0066] Determine a first target current limiting level according to the current usage capacity and the critical usage capacity;

[0067] The rate of the current data is adjusted according to the current limiting rate corresponding to the first target current limiting level to complete the current limiting processing sent from the host to the cache.

[0068] Specifically, the critical usage capacity of the hard disk is obtained. It should be noted that the critical usage capacity is the maximum allowable data capacity of the hard disk that can be written, and the critical usage capacity here is greater than the capacity threshold. The corresponding first target current limiting level is determined according to the difference between the current usage capacity and the critical usage capacity. Here, a mapping relationship is established in advance between the capacity data (difference), the current limiting level and the current limiting rate corresponding to each current limiting level. The smaller the capacity data, the higher the corresponding current limiting level, and the larger the corresponding current limiting rate. After the difference corresponding to the capacity data is determined, the current limiting level is determined based on the difference corresponding to the capacity data, that is, the first target current limiting level. According to the pre-corresponding mapping relationship, the current limiting rate corresponding to the first target current limiting level is determined. The rate of the current data is adjusted according to the current limiting rate of the first target current limiting level to complete the current limiting processing.

[0069] This embodiment provides a method for determining a first target current limiting level based on the current usage capacity and the critical usage capacity, and adjusting the rate of the current data at the current limiting rate corresponding to the first target current limiting level to complete the current limiting process. Current limiting is achieved by adjusting the rate of data traffic, extending the response time of the host write IO, delaying and reducing the data written to the cache by the host, and simplifying the current limiting operation.

[0070] In some other embodiments, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including:

[0071] Obtain the unit request quantity and the threshold request quantity corresponding to the host sent to the cache; wherein the unit request quantity is greater than the threshold request quantity;

[0072] Determine the number of current limiting requests based on the number of unit requests and the number of threshold requests;

[0073] The data sent from the host to the cache is sent according to the number of current limiting requests to complete the current limiting processing sent from the host to the cache.

[0074] Specifically, when processing business logic, the server will frequently access certain data, so the number of requests from the host is reduced. The number of requests here refers to a smaller value that only allows requests to enter the system per second at the request entry front end. Any more requests will be directly denied service at the entry front end. The unit request number corresponds to the number of requests transmitted per second, that is, the current number of requests entering the system at the request entry front end. Currently, the unit request number is limited by setting the threshold request number, and the unit request number is adjusted to the threshold request number as the current limiting request number. Therefore, the data sent to the cache by the host is sent according to the current limiting request number to complete the current limiting processing.

[0075] This embodiment provides a method for limiting the flow of data sent from the host to the cache by reducing the number of data requests. The flow limiting mechanism is dynamically adjusted according to the number of requests to achieve flow limiting processing of data, protect backend data from being affected by excessive requests, and ensure the stability and responsiveness of the system.

[0076] In some other embodiments, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including:

[0077] Obtain the initial rate and critical current limiting rate of the current data sent from the host to the cache;

[0078] The initial rate of the current data is adjusted according to the critical current limiting rate to complete the current limiting processing sent from the host to the cache.

[0079] Specifically, the current limiting process here is to adjust the initial rate of the current data sent by the host to the cache to the critical current limiting rate in one step, that is, to send the current data according to the critical current limiting rate. The critical current limiting rate here is the minimum rate, which corresponds to the highest level of current limiting.

[0080] This embodiment provides a method of adjusting the initial speed to the critical current limiting rate in one step to achieve the highest level of current limiting, so as to maximize the extension of the host input output (IO) writing response time.

[0081] In step S13, the current stored data in the cache is flushed to the hard disk. In this flushing process, the flushing time is calculated according to the amount of data on the back-end disk and the cache. The determination process of this flushing time is not limited. The flushing time can be determined by the amount of data and the write speed, or according to the configured submission interval, the minimum number of pages submitted each time, and the maximum interval between two submissions. It should be noted that the flushing speed here can also be adjusted to reduce the flushing time.

[0082] Based on the current storage data flushed to the hard disk, the current used capacity of the hard disk is updated.

[0083] In some embodiments, updating the current used capacity of the hard disk includes:

[0084] Obtain the data capacity corresponding to the write data and the delete data of the current storage data in the cache;

[0085] The current used capacity of the hard disk and the data capacity of the written data are added to obtain a first data capacity;

[0086] The first data capacity is subtracted from the data capacity of the deleted data to obtain the updated current usage capacity of the hard disk.

[0087] Specifically, considering that the currently stored data includes write data and delete data, it is necessary to obtain the data capacity corresponding to the write data and delete data of the currently stored data, and determine the current usage capacity of the updated hard disk based on the current usage capacity of the hard disk, the data capacity of the write data, and the data capacity of the delete data.

[0088] The current used capacity of the hard disk is added to the data capacity of the written data to obtain the first data capacity; the first data capacity is subtracted from the data capacity of the deleted data to obtain the updated current used capacity of the hard disk.

[0089] The present embodiment provides an update of the current usage capacity of the hard disk based on the cached current storage data. Considering that after the cached current storage data is flushed to the hard disk, the final storage data actually stored in the hard disk is counted as the current usage capacity of the updated hard disk, so as to facilitate the final determination of the volume mounting mode according to the updated current usage capacity of the hard disk and the capacity threshold, so that the real volume mounting mode is determined by fully considering the original usage capacity of the hard disk and the cached storage data.

[0090] When updating the current used capacity of the hard disk, determine the volume mount mode based on the updated current used capacity of the hard disk and the capacity threshold to achieve real-time capacity acquisition. After refreshing the disk, check whether the read-only condition of the volume is met. If so, set it to read-only mode. If not, continue to maintain the volume in read-write mode.

[0091] In some embodiments, in an all-flash system, the use of volume capacity is counted at the pool granularity. As the volume is used, more and more data is written. When the pool capacity reaches the read-only condition, the increase and decrease of the storage pool capacity is uncertain because the write cache module executes insert I (L1P1), insert I (P1L1), and then delete D (P0L0) in an uncertain manner. Therefore, after the capacity reaches the read-only condition, the pool capacity is not set to read-only. At this time, the storage device is limited to the highest level of host response flow, and the storage slows down the response to the host IO. This will have a back pressure effect on the host, and the number of input / output operations per second (Input / Output Operations Per Second) sent by the host will increase. Second, IOPS is controlled in a very low range, and a signal flag is set at the same time. The drive cache and write cache are started to flush the disk immediately. The capacity is obtained in real time before the disk is flushed. After the disk is flushed, see whether the pool reaches read-only at this time; if it reaches read-only, set the pool status to read-only; if not, set the current limiting condition again according to the capacity, and clear the flag; after the read-only condition is met, execute the above steps again until the pool actually reaches the read-only condition and sets the pool status to read-only.

[0092] The embodiment of the present invention provides a method for determining storage capacity based on a hard disk, which obtains the current usage capacity of the hard disk and the capacity threshold visible to the outside; when the current usage capacity is greater than the capacity threshold, the data sent by the host to the cache is subjected to flow limiting processing to obtain the data after flow limiting and sent to the cache; the current storage data of the cache is flushed to the hard disk, and the current usage capacity of the hard disk is updated, so as to determine the mounting mode of the volume according to the current usage capacity and capacity threshold of the updated hard disk. On the one hand, when the current usage capacity of the hard disk is greater than the capacity threshold, the present invention will not immediately switch the mounting mode of the volume, but will perform flow limiting processing on the data sent by the corresponding host to the cache, so as to extend the response time of the host writing IO by reducing the amount of data processed by flow limiting, and at the same time, before the mounting mode of the volume is changed to read-only mode, the data written to the cache by the host is delayed and reduced by flow limiting processing, and the time for the cache to count the current storage data is given, so as to flush it to the hard disk, so as to avoid the failure of writing the data in the corresponding cache of the host when the volume is switched to read-only mode, thereby improving the security of the data. On the other hand, the cached current storage data is flushed to the hard disk. Considering that the cached storage data includes write data and delete data, the final storage data actually stored in the hard disk is counted as the current usage capacity of the updated hard disk, and the volume mounting mode is finally determined based on the current usage capacity of the updated hard disk and the capacity threshold. This is to avoid the problem of reduced user experience caused by a large difference between the storage data visible to the user and the actual storage capacity of the hard disk. The present invention fully considers that when the deleted data in the memory cache is greater than the written data, the corresponding current usage capacity of the updated hard disk will be less than the current usage capacity of the hard disk before the update, so as to improve the user experience by making the mounting mode of the volume prematurely downloaded into read-only mode.

[0093] In some embodiments, flushing the cached current storage data to the hard disk includes:

[0094] Obtain the cache reserved space of the hard disk and the write speed of the hard disk; wherein the storage data capacity of the cache reserved space is greater than the storage data capacity of the cache; and the rate corresponding to the write speed of the hard disk is greater than the corresponding current limiting rate after the host sends it to the cache;

[0095] Receive the current storage data of the cache;

[0096] Determine the hard disk refresh time based on the current storage data and write speed;

[0097] The currently stored data is flushed to the cache reserved space of the hard disk according to the flush time.

[0098] Specifically, the cache reserved space of the hard disk is obtained. The cache reserved space (Over-Provisioning, OP) here is the capacity part that cannot be operated by users. Its main function is that this part of the space is invisible to users and is used to improve the performance and durability of the SSD. Through the reserved space, the SSD controller can better manage the wear of the flash memory (Not AND, NAND) during its service life, thereby extending the service life of the hard disk.

[0099] Get the hard disk write speed. At this time, the storage data capacity of the cache reserved space is greater than the storage data capacity of the cache. In order to complete the disk flushing as soon as possible, to prevent the host from sending the limited rate to the cache faster, the corresponding cache starts to store more new data. Therefore, the rate corresponding to the hard disk write speed here is greater than the corresponding limited rate after the host sends it to the cache.

[0100] The hard disk refresh time is determined according to the current storage data and write speed of the cache, that is, current storage data / write speed=flush time. The current storage data is flushed to the cache reserved space of the hard disk based on the flush time.

[0101] The present embodiment provides a flush time determined by the current write speed, and controls the amount of writes to the write cache according to the flush time, thereby ensuring that the capacity of the flush cache does not exceed the cache reserved space of the hard disk, protecting data from being overwritten, and improving data security.

[0102] In some other embodiments, flushing the cached current storage data to the hard disk includes:

[0103] Get the cache reserved space of the hard disk and the initial write speed of the hard disk;

[0104] Determine the amount of written data and the amount of deleted data of the currently stored data in the cache;

[0105] If the amount of deleted data is greater than the amount of written data, the initial writing speed is adjusted according to the speed adjustment step to obtain an adjusted writing speed;

[0106] Determine the flush time of the hard disk based on the adjusted write speed and the current storage data of the cache;

[0107] The currently stored data is flushed to the cache reserved space of the hard disk according to the flush time.

[0108] Specifically, considering the reduction of the hard disk's flushing time, it is necessary to speed up the hard disk's write speed. The condition for speeding up the hard disk's write speed here is based on the comparison between the amount of write data and the amount of deleted data of the cached current storage data. If the amount of deleted data is greater than the amount of written data, the initial write speed is adjusted according to the adjustment speed step to obtain the adjusted write speed. Considering that when the amount of deleted data is greater than the amount of written data, the current usage capacity of the corresponding hard disk will be reduced, therefore, it is necessary to speed up the flushing speed to complete it as soon as possible.

[0109] The flush time of the hard disk is determined based on the adjusted write speed and the current storage data in the cache, and the current storage data is flushed to the cache reserved space of the hard disk according to the flush time. The determination of the flush time and the flushing process to the cache reserved space of the hard disk are the same as those in the above embodiment and will not be repeated here.

[0110] This embodiment provides that when the amount of deleted data is greater than the amount of written data, the initial write speed is adjusted according to the adjustment speed step to obtain an adjusted write speed, thereby improving the refresh time of the hard disk and ensuring data security.

[0111] In some embodiments, determining the volume mount mode according to the updated current usage capacity of the hard disk and the capacity threshold includes:

[0112] If the current used capacity of the updated hard disk is greater than or equal to the capacity threshold, the mount mode of the volume is determined to be read-only mode;

[0113] If the current used capacity of the updated hard disk is less than the capacity threshold, the mount mode of the volume is determined to be read-write mode;

[0114] Correspondingly, after determining the volume mount mode, it also includes:

[0115] When the volume is mounted in read-only mode, switch the read-write business mode to query mode;

[0116] When the mount mode of the volume is the read-write mode, the current used capacity of the updated hard disk is subtracted from the capacity threshold to obtain the second data capacity;

[0117] Pre-acquire a mapping relationship between a data capacity range and a current limiting level; wherein the data capacity range and the current limiting level are in a one-to-one mapping relationship;

[0118] Determining a target data capacity range corresponding to the second data capacity in the mapping relationship;

[0119] Determine a corresponding second target current limiting level according to the target data capacity range;

[0120] The limited data is sent to the cache according to the limited rate corresponding to the second target limited level.

[0121] Specifically, with respect to the determination of the volume mount mode, if the current used capacity of the updated hard disk is greater than or equal to the capacity threshold, the volume mount mode is determined to be read-only mode; if the current used capacity of the updated hard disk is less than the capacity threshold, the volume mount mode is determined to be read-write mode.

[0122] After determining the volume mount mode, if it is read-only mode, the host's read-write business mode is switched to query mode, and only query services are provided externally. If it is read-write mode, it means that the current used capacity of the updated hard disk does not exceed the capacity threshold visible to the outside. In order to prevent the host from sending data to the cache according to the original data flow, resulting in the actual used capacity of the hard disk exceeding the capacity threshold visible to the outside when the subsequent cached data is flushed to the hard disk, at this time, it is necessary to perform flow control when the current capacity threshold is not exceeded.

[0123] The current used capacity of the updated hard disk is subtracted from the capacity threshold to obtain the second data capacity. A mapping relationship exists between the second data capacity, the data capacity range, and the current limiting level, and the data capacity range and the current limiting level are in a one-to-one mapping relationship. The mapping relationship here may be the same as the embodiment determined by the first target current limiting level in the above embodiment, or it may be different, and is not limited here. In addition, the current limiting rate corresponding to different current limiting levels may be smaller than the current limiting rate corresponding to the first target current limiting level in the above embodiment. Considering that the capacity threshold visible to the outside is not currently exceeded, a preventive current limiting processing operation is performed.

[0124] Here, the target data capacity range corresponding to the second data capacity is determined based on the mapping relationship, and the corresponding second target current limiting level is determined based on the target data capacity range, and the limited data (here corresponding to the limited data in step S12) is sent to the cache at the corresponding current limiting rate.

[0125] This embodiment provides a process for determining the volume mount mode, and in the corresponding read-only mode, changes the host's read-write business to a query business, and in the corresponding read-write mode, performs preventive flow limiting processing to improve the user experience.

[0126] When the capacity of the pool reaches the read-only condition (the used capacity of the volume on the pool is greater than the capacity displayed by the pool externally), the flow is limited. There is space reserved for OP in the storage system, and this OP space will exceed the capacity of the cache. The flush time is calculated according to the amount of data on the back-end disk and the cache. The amount of write cache written is controlled according to this time, which can ensure that the capacity of the entire cache does not exceed the space reserved for OP. The OP space can undertake the data in the cache. However, it is also necessary to ensure that after the capacity of the pool to provide services externally is reached, it is displayed externally in a read-only state to protect the data from being overwritten or not being written to the disk in the cache, to prevent data loss after the device loses power.

[0127] When the pool capacity reaches the read-only condition, a flag is set. When the read-only condition is reached, the cache module and write cache module are immediately driven to flush the disk. At this time, the host's write IO response time will be extended, but it can still provide services to the outside world, and will not cause downtime of the host business. After flushing the disk, obtain the capacity again to see if it reaches read-only at this time; if it reaches read-only, set the pool status to read-only; if not, set the current limiting conditions according to the capacity, and set the current limiting to a reasonable range. If the read-only condition is really reached, the status of all volumes on the pool will be set to read-only. After read-only, the volume can only provide query services to the outside world.

[0128] This can make capacity statistics more accurate and avoid the situation where the pool is read-only even though the read-only condition is not met.

[0129] Figure 2 A schematic diagram of a fast disk flushing of cached data provided by an embodiment of the present invention, such as Figure 2 As shown in the figure, when the metadata write cache module capacity is full, the flag is set, the driver cache and write cache flush data, and the capacity is recalculated. According to the current capacity, it is decided whether to set the volume on the pool to read-only state.

[0130] Furthermore, the present invention also provides a storage system, comprising a host and a hard disk; wherein the cache is located on the host;

[0131] The host is used to execute the steps of the above-mentioned method for determining the storage capacity based on the hard disk.

[0132] For an introduction to a storage system provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above hard disk-based storage capacity determination method.

[0133] The above describes in detail various embodiments corresponding to the method for determining storage capacity based on a hard disk. On this basis, the present invention also discloses a device for determining storage capacity based on a hard disk corresponding to the above method. Figure 3The structure diagram of a storage capacity determination device based on a hard disk provided by an embodiment of the present invention. Figure 3 As shown, the storage capacity determination device based on the hard disk includes:

[0134] The acquisition module 11 is used to acquire the current used capacity of the hard disk and the capacity threshold visible to the outside;

[0135] The current limiting processing module 12 is used to perform current limiting processing on the data sent from the host to the cache to obtain the current-limited data and send it to the cache when the current usage capacity is greater than the capacity threshold;

[0136] The flush processing module 13 is used to flush the cached current storage data to the hard disk and update the current used capacity of the hard disk, so as to determine the volume mounting mode according to the updated current used capacity of the hard disk and the capacity threshold.

[0137] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the method part for the embodiments of the device part, and will not be repeated here.

[0138] For an introduction to a storage capacity determination device based on a hard disk provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above method for determining storage capacity based on a hard disk.

[0139] Figure 4 A structural diagram of a server provided in an embodiment of the present invention, such as Figure 4 As shown, the server includes:

[0140] A memory 21, used for storing computer programs;

[0141] The processor 22 is used to implement the steps of the method for determining the storage capacity based on the hard disk when executing the computer program.

[0142] Among them, the processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 22 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 22 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0143] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used to store the following computer program 211, wherein, after the computer program is loaded and executed by the processor 22, it can implement the relevant steps of the hard disk-based storage capacity determination method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 212 may include Windows, Unix, Linux, etc. The data 213 may include, but is not limited to, data involved in the hard disk-based storage capacity determination method, etc.

[0144] In some embodiments, the server may further include a display screen 23 , an input / output interface 24 , a communication interface 25 , a power supply 26 , and a communication bus 27 .

[0145] Those skilled in the art can understand that Figure 4 The structure shown in the figure does not constitute a limitation on the server, and may include more or fewer components than shown in the figure.

[0146] The processor 22 implements the hard disk-based storage capacity determination method provided in any of the above embodiments by calling instructions stored in the memory 21 .

[0147] For an introduction to a server provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above hard disk-based storage capacity determination method.

[0148] Furthermore, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor 22, the steps of the above-mentioned method for determining storage capacity based on a hard disk are implemented.

[0149] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0150] For an introduction to a computer-readable storage medium provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above hard disk-based storage capacity determination method.

[0151] Furthermore, the present invention also provides a computer program product, including a computer program / instruction, which implements the steps of the hard disk-based storage capacity determination method when executed by a processor.

[0152] For an introduction to a computer program product provided by the present invention, please refer to the above method embodiment, which will not be described in detail herein. It has the same beneficial effects as the above hard disk-based storage capacity determination method.

[0153] The above is a detailed introduction to a storage capacity determination method based on a hard disk, a storage system, a server and a medium provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

[0154] It should also be noted that, in this specification, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A method for determining storage capacity based on a hard disk, characterized in that: include: Get the current used capacity of the hard disk and the capacity threshold visible to the outside world; When the currently used capacity is greater than the capacity threshold, the data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data and then sent to the cache; The cached current storage data is flushed to the hard disk, and the current used capacity of the hard disk is updated, so as to determine the volume mounting mode according to the updated current used capacity of the hard disk and the capacity threshold.

2. The method for determining storage capacity based on a hard disk according to claim 1, characterized in that: The data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including: Acquire the critical usage capacity of the hard disk and the rate at which the host sends current data to the cache; wherein the critical usage capacity is greater than the capacity threshold; Determine a first target current limiting level according to the current usage capacity and the critical usage capacity; The rate of the current data is adjusted according to the current limiting rate corresponding to the first target current limiting level to complete the current limiting processing sent by the host to the cache.

3. The method for determining storage capacity based on a hard disk according to claim 1, characterized in that: The data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including: Obtaining the unit request quantity and the threshold request quantity corresponding to the cache sent by the host; wherein the unit request quantity is greater than the threshold request quantity; Determine the number of current limiting requests according to the number of unit requests and the number of threshold requests; The data sent by the host to the cache is sent according to the number of current limiting requests to complete the current limiting processing sent by the host to the cache.

4. The method for determining storage capacity based on a hard disk according to claim 1, characterized in that: The data sent from the host to the cache is subjected to flow limiting processing to obtain the flow-limited data, including: Obtaining an initial rate and a critical current limiting rate of current data sent by the host to the cache; The initial rate of the current data is adjusted according to the critical current limiting rate to complete the current limiting processing sent by the host to the cache.

5. The method for determining storage capacity based on a hard disk according to any one of claims 2 to 4, characterized in that: Flushing the cached current storage data to the hard disk includes: Obtain the cache reserved space of the hard disk and the write speed of the hard disk; wherein the storage data capacity of the cache reserved space is greater than the storage data capacity of the cache; and the rate corresponding to the write speed of the hard disk is greater than the corresponding current limiting rate after the host sends it to the cache; receiving the currently stored data of the cache; Determining a refresh time of the hard disk according to the current storage data and the write speed; The currently stored data is flushed to the cache reserved space of the hard disk according to the flush time.

6. The method for determining storage capacity based on a hard disk according to claim 5, characterized in that: Updating the currently used capacity of the hard disk, including: Obtaining data capacities corresponding to write data and delete data of the currently stored data in the cache; Adding the current used capacity of the hard disk to the data capacity of the written data to obtain a first data capacity; The first data capacity is subtracted from the data capacity of the deleted data to obtain the updated current usage capacity of the hard disk.

7. The method for determining storage capacity based on a hard disk according to claim 6, characterized in that: Determining the volume mounting mode according to the updated current usage capacity of the hard disk and the capacity threshold includes: If the current used capacity of the hard disk after update is greater than or equal to the capacity threshold, determining that the mount mode of the volume is read-only mode; If the current used capacity of the hard disk after update is less than the capacity threshold, determining that the mount mode of the volume is a read-write mode; Correspondingly, after determining the volume mount mode, it also includes: When the volume is mounted in read-only mode, switch the read-write business mode to query mode; When the mount mode of the volume is the read-write mode, subtracting the updated current used capacity of the hard disk from the capacity threshold to obtain a second data capacity; Pre-acquire a mapping relationship between a data capacity range and a current limiting level; wherein the data capacity range and the current limiting level are in a one-to-one mapping relationship; Determining a target data capacity range corresponding to the second data capacity in the mapping relationship; Determine a corresponding second target current limiting level according to the target data capacity range; The limited data is sent to the cache according to the limited rate corresponding to the second target limited level.

8. A storage system, characterized in that: It includes a host and a hard disk; wherein the cache is located on the host; The host is used to execute the steps of the hard disk-based storage capacity determination method as described in any one of claims 1 to 7.

9. A server, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the hard disk-based storage capacity determination method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the hard disk-based storage capacity determination method according to any one of claims 1 to 7 are implemented.

Images