Data reclamation method and apparatus, computing device and computer-readable storage medium

Abstract

Provided in the embodiments of the present description are a data reclamation method and apparatus, a computing device and a computer-readable storage medium. The data reclamation method comprises: acquiring state indicators of a storage unit, the state indicators comprising a utilization rate indicator and a garbage data proportion indicator of the storage unit; according to the utilization rate indicator, determining a storage pressure level; acquiring a data reclamation pressure level classification policy on the basis of the storage pressure level, the data reclamation pressure level classification policy comprising a level classification parameter used for determining data reclamation pressure levels, wherein the higher a storage pressure level, the higher a data reclamation pressure level determined by the level classification parameter; according to the garbage data proportion indicator and the data reclamation pressure level classification policy, determining a data reclamation pressure level; and, on the basis of the data reclamation pressure level, predicting a data reclamation bandwidth of the storage unit. The present disclosure ensures the rationality of data reclamation bandwidth setting, thus remarkedly improving the data reclamation performance and the read-write performance.

Description

Data recovery method, device, computing device, and computer-readable storage medium

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on November 9, 2023, with application number 202311499840.2 and application name “Data Recovery Method, Apparatus, Computing Device and Computer-Readable Storage Medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a data recovery method, apparatus, computing device, and computer-readable storage medium. Background Art

[0003] In data storage scenarios, data is usually stored using disks. During disk use, continuous data recovery is required to free up disk space.

[0004] The input / output (I / O) capacity of disks is limited. A high data recovery throughput will significantly affect the user's data read and write performance. If the data recovery throughput is too low, space will not be released in a timely manner. Therefore, a method for reasonably setting the data recovery bandwidth is urgently needed to utilize the set data recovery bandwidth for data recovery, thereby reducing the impact on read and write performance.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a data recovery method, apparatus, computing device, and computer-readable storage medium to address technical deficiencies in the prior art.

[0007] According to a first aspect of an embodiment of this specification, a data recovery method is provided, comprising:

[0008] Obtain a status indicator of a storage unit, wherein the status indicator includes a utilization indicator and a garbage data ratio indicator of the storage unit; determine a storage pressure level based on the utilization indicator; obtain a data recovery pressure level division strategy based on the storage pressure level, wherein the data recovery pressure level division strategy includes a level division parameter for determining the data recovery pressure level, and in the case where the storage pressure level is higher, the data recovery pressure level determined by the level division parameter is higher; determine the data recovery pressure level based on the garbage data ratio indicator and the data recovery pressure level division strategy; and predict the data recovery bandwidth of the storage unit based on the data recovery pressure level.

[0009] According to a second aspect of an embodiment of this specification, a data recovery device is provided, comprising:

[0010] The first acquisition module is configured to acquire a status indicator of the storage unit, wherein the status indicator includes a utilization indicator and a garbage data ratio indicator of the storage unit; the first determination module is configured to determine a storage pressure level based on the utilization indicator; the second acquisition module is configured to acquire a data recovery pressure level division strategy based on the storage pressure level, wherein the data recovery pressure level division strategy includes a level division parameter for determining the data recovery pressure level, and in the case where the storage pressure level is higher, the data recovery pressure level determined by the level division parameter is higher; the second determination module is configured to determine the data recovery pressure level based on the garbage data ratio indicator and the data recovery pressure level division strategy; the prediction module is configured to predict the data recovery bandwidth of the storage unit based on the data recovery pressure level.

[0011] According to the third aspect of the embodiments of this specification, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data recovery method.

[0012] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data recovery method are implemented.

[0013] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data recovery method.

[0014] One embodiment of the present specification obtains a status indicator of a storage unit, wherein the status indicator includes a utilization indicator and a garbage data ratio indicator of the storage unit; determines a storage pressure level based on the utilization indicator; obtains a data recovery pressure level division strategy based on the storage pressure level, wherein the data recovery pressure level division strategy includes a level division parameter for determining the data recovery pressure level, and the higher the storage pressure level, the higher the data recovery pressure level determined by the level division parameter; determines the data recovery pressure level based on the garbage data ratio indicator and the data recovery pressure level division strategy; and predicts the data recovery bandwidth of the storage unit based on the data recovery pressure level. Based on the utilization indicator in the status indicator of the storage unit, the storage pressure level is divided, and based on the storage pressure level obtained by the division, a data recovery pressure division strategy corresponding to the storage pressure is obtained, so that when the data recovery pressure level is divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure level, that is, the division is performed based on the dual-dimensional utilization indicator and garbage data ratio indicator, so that the utilization rate and garbage data ratio of the storage unit are simultaneously taken into account when determining the data recovery bandwidth, ensuring the rationality of the data recovery bandwidth setting, thereby significantly improving both data recovery performance and read and write performance.

[0015] These and other aspects of the present disclosure will become more apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] FIG1a is a schematic diagram of a state machine of a Zone in a data recovery method provided by an embodiment of this specification;

[0018] FIG1b is a schematic diagram of a data recovery process in a data recovery method provided in one embodiment of this specification;

[0019] FIG1c is a schematic diagram showing the read and write performance in a high-water mark, garbage-free data scenario in a current data recovery method;

[0020] Figure 1d is a schematic diagram of the fluctuation amplitude of single-machine throughput in a current data recovery method;

[0021] FIG1e is a schematic diagram of data recovery flow fluctuations in a current data recovery method;

[0022] FIG1f is a schematic diagram showing the fluctuation of the impact of data recovery on write throughput in a current data recovery method;

[0023] FIG1g is a schematic diagram of a trigger frequency of data recovery in a current data recovery method;

[0024] FIG1h is a schematic diagram of a data recovery flow control value in a current data recovery method;

[0025] FIG2 is a schematic diagram of an interaction process under a data recovery system architecture provided by an embodiment of this specification;

[0026] FIG3 is a framework diagram of a data recovery system provided by one embodiment of this specification;

[0027] FIG4 is a flow chart of a data recovery method provided by one embodiment of this specification;

[0028] FIG5 is a flowchart of a data recovery method according to an embodiment of the present disclosure;

[0029] FIG6 is a schematic diagram of system module interaction in a data recovery method provided by one embodiment of the present invention;

[0030] FIG7 a is a schematic diagram of a single-machine throughput in a data recovery method provided in one embodiment of this specification under high storage pressure and without data recovery;

[0031] FIG7 b is a schematic diagram of data recovery in a data recovery method provided in an embodiment of this specification under high storage pressure and no data recovery scenario;

[0032] FIG7 c is a schematic diagram of a single-machine throughput in a current data recovery method under high storage pressure and no data recovery scenario;

[0033] FIG7 d is a schematic diagram of data recovery in a current data recovery method under high storage pressure and no data recovery scenario;

[0034] FIG7e is a schematic diagram of a single-machine throughput under normal storage pressure and high garbage data scenarios in a data recovery method provided by one embodiment of this specification;

[0035] FIG7f is a schematic diagram of single-machine throughput under normal storage pressure and high garbage data scenario in another data recovery method provided by one embodiment of this specification;

[0036] FIG7g is a schematic diagram of a data recovery bandwidth change in a data recovery method provided by an embodiment of this specification;

[0037] FIG8 is a schematic structural diagram of a data recovery device provided by an embodiment of this specification;

[0038] FIG9 is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0039] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0040] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0042] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0043] First, the terms involved in one or more embodiments of this specification are explained.

[0044] Standalone storage engine: In a distributed storage system, a functional node is responsible for managing the storage of user data on storage media. In a Pangu cluster, a storage server deploys a standalone storage engine process, which is responsible for managing multiple disks on the server.

[0045] Shingled Magnetic Recording Hard-Disk Drive (SMR HDD): A high-capacity disk that uses shingled magnetic recording technology. SMR disks partially overlap the data tracks on the disk.

[0046] Zone: The smallest processing unit on an SMR disk, where the smallest processing unit is the smallest allocation and erasure unit. For example, exemplary characteristics of a Zone are shown in Table 1 below: Zone Characteristics Record Table.

[0047] Table 1: Zone characteristic record table

[0048] Referring to FIG1a, FIG1a shows a schematic diagram of a state machine of a zone in a data recovery method provided by an embodiment of the present specification. A zone includes four states: garbage state (Garbage), free state (Free), open state (Open), and closed state (Closed); the garbage state is characterized by no valid data and waiting to be recovered, the free state is characterized by being allocable and not readable or writable, the open state is characterized by being appendable and sequentially / randomly readable, and the closed state is characterized by being not writable and sequentially / randomly readable; the four states can form a closed loop through corresponding operations. When the state of the zone is in the free state, an open operation is performed, and the state of the zone can be converted to the open state. When the state of the zone is in the open state, a close operation is performed, and the state of the zone can be converted to the closed state. When the state of the zone is in the closed state, a valid data deletion operation is performed, and the state of the zone can be converted to the garbage state. When the state of the zone is in the garbage state, an erase operation is performed, and the state of the zone can be converted to the free state.

[0049] Data recovery: Read valid data from the source zone and write it to another destination zone with free available space, freeing up the source zone space and retaining invalid data. The source zone is generally a zone with a high proportion of junk data to reduce the amount of valid data read and moved. The destination zone is generally an idle zone. See Figure 1b for details. Figure 1b shows a schematic diagram of the data recovery process in a data recovery method provided in an embodiment of this specification.

[0050] In data storage scenarios, data is typically stored on disks. During disk use, continuous data recycling is required to free up disk space. Disks have limited input / output (I / O) capabilities. High data recycling throughput can significantly impact user data read and write performance. However, low data recycling throughput can delay space release. Therefore, it's necessary to set a garbage collection bandwidth to ensure that data is recycled within a reasonable bandwidth.

[0051] Currently, there are many ways to set the data recovery bandwidth. The first is to set the data recovery bandwidth based on the usage rate of the zone. The higher the zone usage rate, the higher the threshold of the data recovery bandwidth is set. For example, see Figure 1c, which shows a schematic diagram of the read and write performance in a high-water mark no-garbage data scenario in a current data recovery method. The horizontal axis on the left side of the figure is the time axis, and the time includes "12:15, ..., 13:45, 14:00" and so on, and the vertical axis (the unit of the Y axis is percentage) includes "0, 25, 50, 75", the valid data watermark is 95%, the zone utilization rate is 95%, and the garbage data ratio is 0, demonstrating a high-watermark scenario with no garbage data and almost no garbage data requiring collection. The horizontal axis on the right side of the figure is the time axis, including times such as "12:15, ..., 13:45, and 14:00", and the vertical axis (Y-axis in GB / second) includes bandwidth, including "0, 1, and 2". During the write operation at 13:50, the foreground throughput started at 2.68 GB / second and then experienced a significant drop, ultimately leading to a write timeout. This is because the current data recycling bandwidth setting is calculated only based on zone utilization. When the zone utilization rate is above 95%, the foreground write throughput is limited to a minimum of 30 MB / second.

[0052] The second method is shown in Figures 1d and 1e. Figure 1d shows a schematic diagram of the fluctuation amplitude of the throughput of a single machine in a current data recovery method, wherein the horizontal axis is the time axis, and the time includes "14:00, 14:10, 14:20, 14:30, 14:40, 14:50, 15:00", and the vertical axis reads the throughput bandwidth. The unit of the throughput bandwidth is bit / second, and the bandwidth includes 0, 50, 100, and 150. It can be determined that the throughput of a single machine fluctuates periodically around 150M and 100M, and the fluctuation amplitude is about 30%; Figure 1e shows a schematic diagram of the data recovery traffic fluctuation in a current data recovery method The horizontal axis represents the timeline, with the times being 2:00 PM, 2:10 PM, 2:20 PM, 30 PM, 40 PM, 50 PM, and 3:00 PM. The vertical axis represents the garbage collection bandwidth, measured in MB / s. The bandwidth values ​​are 0, 10, 20, 30, 40, 50, and 60 MB / s, including those for Garbage Collection 1 and Garbage Collection 2. The fluctuations in the garbage collection bandwidth along this axis indicate that performance fluctuations are due to periodically triggered data collection traffic. Data collection, in turn, is caused by users periodically deleting files, creating garbage space on SMR disks that needs to be reclaimed. According to the current throttling policy, the data collection bandwidth is set at a constant 50 MB. This rapidly increased data collection traffic to nearly 50 MB / s, significantly impacting user requests.

[0053] The third method is to adapt SMR HDD in the file system and trigger data recycling based on the amount of garbage data. However, there is no precise restriction on the data recycling bandwidth. As long as data recycling is triggered, data recycling is always performed at full bandwidth, so it has a significant impact on the user's foreground reading and writing. See Figure 1f for details. Figure 1f shows a schematic diagram of the fluctuation of data recycling on write throughput in a current data recycling method, where the horizontal axis is the date and time axis, and the date and time include "08 / 25 18:00, 08 / 26 00:00, 08 / 26 06:00, 08 / 26 12:00, 08 / 26 18:00, 08 / 27 00:00, 08 / 27 06:00", and the vertical axis is the bandwidth of write throughput, including "0, 1.4, 2.3, 3.2, 4.1, 5.0". In Figure 1f, when the file system data recycling is started, the user's foreground write throughput is reduced by more than 50%; see Figure 1g, Figure 1g shows a current method Figure 1. Schematic diagram of the data collection trigger frequency in the data collection method. The horizontal axis represents the date and time axis, including "18:00 on 08 / 25, 00:00 on 08 / 26, 06:00 on 08 / 26, 12:00 on 08 / 26, 18:00 on 08 / 26, 00:00 on 08 / 27, and 06:00 on 08 / 27." The vertical axis represents the garbage collection trigger frequency, including "0, 8000, 16000, 24000, 32000, and 4000." Specifically, garbage collection is continuously triggered starting from a single point (12:00 on 08 / 26) without any further control strategy. That is, when garbage collection requires garbage collection, the maximum data collection bandwidth is directly activated for data collection. Even when only a small amount of garbage needs to be collected, or when garbage collection is constantly flowing, the maximum data collection bandwidth is consistently maintained for data collection, significantly impacting the front-end write performance.

[0054] The fourth method is to use the Zone usage rate as the only factor driving the flow control threshold, and drive the data recovery bandwidth threshold through a single factor, that is, the setting of the data recovery bandwidth is based on this single factor only. For details, see Figure 1h. Figure 1h shows a schematic diagram of the data recovery flow control value in a current data recovery method. The horizontal axis is the garbage ratio, the unit is percentage, including "80, ..., 100", and the vertical axis is the recovery bandwidth, the unit is MB / s, including "0, 20, ..., 120". The dotted part in the figure is the recovery bandwidth of garbage recovery, among which, when the garbage ratio is 80, the recovery bandwidth is 8; when the garbage ratio is 81, the recovery bandwidth is 11; when the garbage ratio is 82, the recovery bandwidth is 14; when the garbage ratio is 83, the recovery bandwidth is 18; and when the garbage ratio is 0, 20, ..., 120, the recovery bandwidth is 120. When the garbage ratio is 84, the recycling bandwidth is 21; when the garbage ratio is 85, the recycling bandwidth is 25; when the garbage ratio is 86, the recycling bandwidth is 28; when the garbage ratio is 87, the recycling bandwidth is 32; when the garbage ratio is 88, the recycling bandwidth is 35; when the garbage ratio is 89, the recycling bandwidth is 43; when the garbage ratio is 90, the recycling bandwidth is 47; when the garbage ratio is 91, the recycling bandwidth is 53; when the garbage ratio is 92, the recycling bandwidth is 60; when the garbage ratio is 93, the recycling bandwidth is 60; when the garbage ratio is 94, the recycling bandwidth is 60; when the garbage ratio is 95, the recycling bandwidth is 100; when the garbage ratio is 96, the recycling bandwidth is 100. This solution only considers the zone utilization rate when setting the data recycling bandwidth, which seriously affects the performance of foreground writes.

[0055] Therefore, there is an urgent need for a method for reasonably setting the data recovery bandwidth so as to utilize the set data recovery bandwidth for data recovery, thereby reducing the impact on read and write performance.

[0056] In an embodiment of the present specification, a data recovery method for reasonably setting the data recovery bandwidth is provided. Specifically, storage pressure levels are divided according to the utilization index in the status index of the storage unit. Based on the storage pressure levels obtained by the division, a data recovery pressure division strategy corresponding to the storage pressure is obtained. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two dimensions of the utilization index and the garbage data ratio index. This realizes that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and read and write performance.

[0057] In this specification, a data recovery method is provided. This specification also relates to a data recovery device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0058] Refer to Figure 2, which shows a schematic diagram of the interactive process under a data recovery system architecture provided by an embodiment of this specification. As shown in Figure 2, the system includes a storage server and a client. A stand-alone storage engine is deployed on the storage server. The stand-alone storage engine is responsible for managing multiple storage disks on the storage server. There are multiple storage units on the storage disk, wherein the storage unit is the smallest processing unit for data recovery processing in the storage disk.

[0059] Based on the demand for data recovery, the client sends a data recovery request to the storage server so that the storage server performs data recovery, wherein the data recovery request can be the request that initially instructs the storage server to start data recovery. After receiving the data recovery request, the storage server automatically predicts the data recovery bandwidth based on the data recovery request, and guides the storage server to perform data recovery according to the predicted data recovery bandwidth. Optionally, the data recovery request includes a window interval for data recovery, so that the storage server predicts the data recovery bandwidth according to the window interval, and performs data recovery within each window interval according to the predicted data recovery bandwidth.

[0060] Storage server: used to obtain the status indicators of the storage unit through a stand-alone storage engine, wherein the status indicators include the utilization indicator and the garbage data ratio indicator of the storage unit; determine the storage pressure level according to the utilization indicator; obtain the data recovery pressure level division strategy based on the storage pressure level, wherein the data recovery pressure level division strategy includes a level division parameter for determining the data recovery pressure level, and the higher the storage pressure level, the higher the data recovery pressure level determined by the level division parameter; determine the data recovery pressure level according to the garbage data ratio indicator and the data recovery pressure level division strategy; and predict the data recovery bandwidth of the storage unit based on the data recovery pressure level.

[0061] Optionally, after predicting the data recovery bandwidth, the storage server guides the storage unit to recover data based on the predicted data recovery bandwidth.

[0062] Optionally, the data recovery performed by the storage server may be a pre-set program in the storage server. When the storage server is running, data recovery can be performed continuously and automatically without a data recovery request from the client.

[0063] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two dimensions of the utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and the read and write performance.

[0064] Refer to Figure 3, which shows a framework diagram of a data recovery system provided by an embodiment of this specification. The system may include a storage server, a stand-alone storage engine deployed on the storage server, the stand-alone storage engine is responsible for managing the functional nodes on the storage medium in the storage server, the functional nodes are multiple storage disks, and a client.

[0065] Users can interact with storage servers through clients to receive data from other clients, or send data to other clients. In data recovery scenarios, users can send data recovery requests to storage servers through clients. Based on the data recovery requests, the storage server obtains the status indicators of the storage units, predicts the data recovery bandwidth based on the status indicators, and pushes the data recovery bandwidth to other clients with which communication has been established.

[0066] The client and storage server are connected via a network. The network provides the medium for the communication link between the client and the storage server. Networks can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by the client may require encoding, transcoding, and compression before being released to the storage server.

[0067] The client can be a browser, an application (APP), or a web application such as a Hypertext Markup Language 5 (H5) application, a light application (also known as a mini-program, a lightweight application), or a cloud application. The client can be developed based on the software development kit (SDK) of the corresponding service provided by the storage server, such as based on the real-time communication (RTC) SDK. The client can be deployed in an electronic device and needs to rely on the device to run or certain APPs in the device to run. For example, the electronic device can have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, tablet computer, personal computer, etc. Various other types of applications can also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0068] Storage servers include those that provide storage services and may also include other services such as servers that provide communication services to multiple clients, servers that support backend training for models used on clients, and servers that process data sent by clients. It should be noted that storage servers can be implemented as distributed server clusters consisting of multiple servers or as a single server. A server can also be a server in a distributed system or a server integrated with a blockchain. A server can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data and artificial intelligence platforms, or intelligent cloud computing servers or intelligent cloud hosts equipped with artificial intelligence technology.

[0069] Referring to FIG. 4 , FIG. 4 shows a flow chart of a data recovery method provided in an embodiment of this specification, which specifically includes the following steps.

[0070] Step 402: Obtain status indicators of the storage unit, wherein the status indicators include a usage rate indicator and a garbage data ratio indicator of the storage unit.

[0071] When there is a need to recycle garbage data, the status indicators of the storage unit are obtained through the stand-alone storage engine deployed on the storage server, so as to predict the data recycling bandwidth based on the utilization indicator and garbage data ratio indicator contained in the status indicator.

[0072] Specifically, a storage unit is a unit in a server used to store data. A storage unit has multiple states, such as free, garbage, open, and closed. Different states correspond to different storage characteristics. A storage unit stores various data, which is further divided into valid and invalid data. In addition to data storage controls, a storage unit also includes available space for data storage. For example, a storage server deploys a stand-alone storage engine process, which manages multiple SMR disks on the storage server. SMR disks contain multiple Zone units that serve as storage units for data storage. Status indicators represent the storage unit's storage performance. These indicators include utilization and garbage data ratio.

[0073] The utilization rate indicator represents the usage of the storage unit. Utilization rate indicators can include space utilization rate indicators and storage unit utilization rate indicators. The space utilization rate indicator is the ratio of valid data in the storage unit to the total space, and the storage unit utilization rate indicator is the ratio of the number of closed storage units to the total number of storage units. The garbage data ratio indicator represents the proportion of garbage data in the storage unit. The garbage data ratio indicator is the proportion of invalid data in the total data storage space of the storage unit. The garbage data ratio indicator is equal to the storage unit utilization rate indicator minus the space utilization rate indicator.

[0074] There are many ways to obtain the status indicators of the storage unit. The status indicators of the storage unit can be collected based on the client's instructions; the status indicators of the storage unit can be collected according to a preset time window; or the client can send the status indicators to the server after collecting the status indicators of the storage unit.

[0075] The data recovery bandwidth is predicted by the utilization rate indicator and the garbage data ratio indicator included in the status indicator, so that the predicted data recovery bandwidth is based on at least two indicators, ensuring the rationality of the data recovery bandwidth prediction and thus ensuring the stability of data recovery and read and write performance.

[0076] Step 404: Determine the storage pressure level according to the usage rate indicator.

[0077] Specifically, a storage pressure classification strategy refers to a strategy for classifying storage pressure. This strategy includes at least a storage pressure classification method and a classification parameter used to classify storage pressure levels. The classification parameter can be a threshold. The utilization index is compared with the threshold, and the storage pressure level is determined based on the comparison result. The storage pressure level refers to the level to which the storage pressure belongs. The determination of the storage pressure level is related to the utilization index included in the acquired status index. The storage pressure level affects the prediction of data reclaim bandwidth.

[0078] The implementation method of determining the storage pressure level according to the usage rate index may be to determine the storage pressure level according to the usage rate index and the storage pressure level classification strategy.

[0079] The implementation method of determining the storage pressure level based on the utilization index and the storage pressure level classification strategy can be to compare the utilization index with the utilization pressure threshold based on the storage pressure level classification strategy; and determine the storage pressure level based on the comparison result; or it can be to input the utilization index and the storage pressure level classification strategy into the storage pressure level classification model to obtain the storage pressure level output by the storage pressure level classification model, wherein the storage pressure level classification model is trained based on multiple sample utilization indicators, sample storage pressure level classification strategies and storage pressure level labels corresponding to the samples.

[0080] An implementation method for comparing the usage index with the usage pressure threshold based on the storage pressure level classification strategy may be to compare the storage unit usage index and the space usage index with the corresponding pressure thresholds respectively based on the storage pressure level classification strategy.

[0081] The implementation method of determining the storage pressure level based on the comparison result can be that when the comparison result is that the usage index is greater than or equal to the usage pressure threshold, the storage pressure level is determined to be the first storage pressure level; when the comparison result is that the usage index is less than the usage pressure threshold, the storage pressure level is determined to be the second storage pressure level.

[0082] The implementation method of determining the storage pressure level as the first storage pressure level when the comparison result is that the usage index is greater than or equal to the usage pressure threshold may be to determine the storage pressure level as the first storage pressure level when the comparison result is that the storage unit usage index is greater than or equal to the first pressure threshold, or the space usage index is greater than or equal to the second pressure threshold.

[0083] When the comparison result is that the usage index is less than the usage pressure threshold, the implementation method of determining the storage pressure level to be the second storage pressure level may be that when the comparison result is that the storage unit usage index is less than the first pressure threshold and the space usage index is less than the second pressure threshold, the storage pressure level is determined to be the second storage pressure level.

[0084] In an optional embodiment of the present specification, the utilization index includes a storage unit utilization index and a space utilization index. Based on the storage unit utilization index, the space utilization index, and the storage pressure level classification strategy, the storage pressure level of the storage unit is determined. That is, the number of storage units is at least one, the utilization index includes the storage unit utilization index and the space utilization index, the storage unit utilization index is the proportion of saturated storage units in at least one storage unit, the saturated storage unit is a storage unit to which data cannot be written but is readable, and the space utilization index is the proportion of valid data in at least one storage unit to the total data storage space. The above steps of determining the storage pressure level based on the utilization index include the following steps:

[0085] Based on the storage pressure level classification strategy, the storage unit utilization index and the space utilization index are compared with the corresponding pressure thresholds, wherein the storage pressure level classification strategy refers to a strategy for classifying storage pressure levels of storage units;

[0086] Based on the comparison results, the storage pressure level is determined.

[0087] Specifically, the storage unit utilization ratio indicator is the ratio of saturated storage units in at least one storage unit to the total number of saturated storage units in the at least one storage unit. For example, if the number of saturated storage units is 2 and the total number of saturated storage units in the at least one storage unit is 7, the storage unit utilization ratio is 2 / 7, and the saturated storage units are the storage units that are in the off state. The total data storage space is the total storage space of the at least one storage unit.

[0088] An implementation method for comparing the storage unit utilization index with the first pressure threshold and the space utilization index with the second pressure threshold based on the storage pressure level classification strategy can be, based on the storage pressure level classification strategy, calculating the difference between the storage unit utilization index and the first pressure threshold, calculating the difference between the space utilization index and the second pressure threshold, and using the two differences as comparison results; or it can be, based on the storage pressure level, calculating the difference between the storage unit utilization index and the first pressure threshold and a benchmark value, respectively, to obtain two first benchmark differences, and calculating the difference between the space utilization index and the second pressure threshold and a basic data, respectively, to obtain two second benchmark differences.

[0089] The storage pressure level may be determined based on the comparison result by determining the storage pressure level based on the calculated difference value; or by determining the storage pressure level based on two first reference differences and two second reference differences.

[0090] By utilizing the storage pressure level division strategy, the storage unit utilization index and the space utilization index are compared with the first pressure threshold and the second pressure threshold respectively, and the storage pressure level of the storage unit is determined based on the comparison results. This allows the data recovery pressure level to be divided based on the determined storage pressure level when the data recovery pressure level is subsequently determined, thereby realizing the dual indicator factors for determining the data recovery bandwidth. That is, on the basis of laying the foundation for the indicator factors for the data recovery bandwidth, the rationality of the data recovery bandwidth prediction is further guaranteed.

[0091] In an optional embodiment of the present specification, determining the storage pressure level based on the comparison result may be based on a comparison result of the storage unit usage index with the first pressure threshold and a comparison result of the space usage index with the second pressure threshold. That is, the above step of determining the storage pressure level based on the comparison result includes the following steps:

[0092] If the comparison result shows that the storage unit usage index is greater than or equal to the first pressure threshold, or the space usage index is greater than or equal to the second pressure threshold, determining the storage pressure level to be the first storage pressure level;

[0093] When the comparison result shows that the storage unit usage index is less than the first pressure threshold and the space usage index is less than the second pressure threshold, the storage pressure level is determined to be the second storage pressure level, which is lower than the first storage pressure level.

[0094] Specifically, the first pressure threshold and the second pressure threshold are thresholds set for level evaluation of utilization indicators. The first pressure threshold corresponds to the storage unit utilization indicator, and the second pressure threshold corresponds to the space utilization indicator. The first pressure threshold and the second pressure threshold can be set as percentages, decimals, etc. Usually, the numerical values ​​of the first pressure threshold and the second pressure threshold are set to be larger, for example, 85%, 95%, etc.

[0095] When the comparison result is that the storage unit utilization index is greater than or equal to the first pressure threshold, or the space utilization index is greater than or equal to the second pressure threshold, the implementation method of determining the storage pressure level as the first storage pressure level may be that when the difference between the storage unit utilization index and the first pressure threshold is greater than or equal to 0, or the difference between the space utilization index and the second pressure threshold is greater than or equal to 0, the storage pressure level is determined to be the first storage pressure level.

[0096] An implementation method for determining the storage pressure level to be the second storage pressure level may be to determine the storage pressure level to be the second storage pressure level when the comparison result is that the storage unit utilization index is less than the first pressure threshold, and the space utilization index is less than the second pressure threshold, and the storage pressure level to be the second storage pressure level when the comparison result is that the difference between the storage unit utilization index and the first pressure threshold is less than 0, and the difference between the space utilization index and the second pressure threshold is less than 0.

[0097] By comparing different results, different storage pressure levels are divided, and different storage pressure levels correspond to different comparison results of usage indicators and thresholds, thereby improving the refined division of storage pressure levels.

[0098] Step 406: Based on the storage pressure level, obtain a data recovery pressure level classification strategy, wherein the data recovery pressure level classification strategy includes a level classification parameter for determining the data recovery pressure level. The higher the storage pressure level, the higher the data recovery pressure level determined by the level classification parameter.

[0099] Specifically, a data recovery pressure grading strategy refers to a strategy for grading data recovery pressure. The data recovery pressure grading strategy includes at least a method for grading data recovery pressure grading and grading parameters used to grading data recovery pressure grading. The grading parameters refer to the benchmark parameters used when grading data recovery pressure grading. The grading parameters can be self-defined benchmark parameters or threshold values.

[0100] The method for obtaining the data recovery pressure level classification strategy based on the storage pressure level may be to obtain the corresponding data recovery pressure level classification strategy from a preset strategy repository based on the storage pressure level.

[0101] When the storage pressure level is higher, in order to release the space of the storage unit as soon as possible, the efficiency requirement for garbage data recovery is higher than that of the low storage pressure level. Therefore, the setting of the equivalent partitioning parameters is more aggressive, that is, when the level partitioning parameters are the same, the higher the data recovery pressure level is determined.

[0102] The higher the storage pressure level, the higher the data recovery pressure level determined by the level division parameters in the corresponding data recovery pressure level division strategy, that is, the storage pressure level affects the data recovery pressure level division, so that the data recovery pressure level division is based on the storage pressure level obtained by the division.

[0103] For example, if the storage pressure level is the first storage pressure level, the level division parameters in the corresponding data recovery pressure level division strategy are 2% and 3%; if the storage pressure level is the second storage pressure level, the level division parameters in the corresponding data recovery pressure level division strategy are 2% and 5%.

[0104] Based on the determined storage pressure level, the corresponding data recovery pressure level division strategy is obtained based on the storage pressure level, so that the data recovery pressure level is determined based on the data recovery pressure level division strategy, and the data recovery pressure level is determined based on the determined storage pressure level.

[0105] Step 408: Determine the data recovery pressure level according to the garbage data ratio indicator and the data recovery pressure level classification strategy.

[0106] Specifically, the data recovery pressure level refers to the level to which the data recovery pressure belongs. The determination of the data recovery pressure level is related to the garbage data ratio indicator included in the acquired status indicator. The data recovery pressure level affects the prediction of the data recovery bandwidth.

[0107] There are many ways to determine the data recovery pressure level based on the garbage data ratio index and the data recovery pressure level classification strategy. The specific method is determined based on actual conditions and is not limited in this manual.

[0108] In one possible implementation of the present specification, the data recovery pressure level is determined based on the garbage data ratio index and the data recovery pressure level classification strategy. The garbage data ratio index can be compared with the level range threshold according to the data recovery pressure level classification strategy, and the data recovery pressure level is determined based on the comparison result.

[0109] In another possible implementation of the present specification, the data recovery pressure level is determined based on the garbage data ratio index and the data recovery pressure level division strategy. The garbage data ratio index and the data recovery pressure level division strategy can be input into a data recovery pressure level division model to obtain the data recovery pressure level output by the data recovery pressure level division model, wherein the data recovery pressure equivalence division model is trained based on multiple sample garbage data ratio indicators, sample data recovery pressure level division strategies and data recovery pressure level labels.

[0110] In an optional embodiment of the present specification, the above steps of determining the data recovery pressure level according to the garbage data ratio index and the data recovery pressure level classification strategy include the following steps:

[0111] According to the data recycling pressure level classification strategy, the garbage data ratio indicator is compared with the level range threshold, where the garbage data ratio indicator is the proportion of invalid data in the storage unit in the total data storage space, and the level range threshold consists of the third pressure threshold and the fourth pressure threshold, and the third pressure threshold is less than the fourth pressure threshold;

[0112] Based on the comparison results, the data recovery pressure level is determined.

[0113] Specifically, a level range threshold refers to a level range consisting of at least two thresholds. For example, if the thresholds are 2% and 3%, then if the thresholds are greater than 2% and less than 3%, then 2% to 3% is determined to be the level range threshold. The third and fourth pressure thresholds are thresholds set to evaluate the level of the junk data ratio indicator. The third and fourth pressure thresholds can be set as percentages, decimals, etc. Usually, the first and second pressure thresholds are set to smaller values, such as 2%, 3%, 5%, etc.

[0114] According to the data recovery pressure level division strategy, the implementation method of comparing the garbage data proportion indicator with the level range threshold can be to analyze the data recovery pressure level division strategy, obtain the level division parameters of the data recovery pressure equivalent division, set the level range threshold based on the level division parameters; and compare the level range threshold with the garbage data proportion indicator.

[0115] The data recovery pressure level division strategy is analyzed and the level division parameters of the data recovery pressure equivalent division are obtained by inputting the data recovery pressure level division strategy into the analytical model and obtaining the level division parameters and level division method of the data recovery pressure equivalent division output by the analytical model, wherein the analytical model is pre-trained based on the sample set.

[0116] The implementation method for determining the data recovery pressure level based on the comparison result may be to analyze the data recovery pressure level classification strategy, obtain the level classification method, and determine the data recovery pressure level based on the comparison result and the level classification method.

[0117] The data recovery pressure level division strategy is analyzed and the level division method is obtained by inputting the data recovery pressure level division strategy into the analysis model to obtain the level division method output by the analysis model, wherein the analysis model is pre-trained based on the sample set.

[0118] According to the data recovery pressure level classification strategy, the garbage data ratio indicator is compared with the level range threshold, and the data recovery pressure level is determined based on the comparison result. Among them, the data recovery pressure level classification strategy is obtained based on the storage pressure level, that is, the data recovery pressure level is divided based on the determined storage pressure level, and the dual indicator factors are realized to determine the data recovery bandwidth. That is, on the basis of laying the indicator factors for the data recovery bandwidth, the rationality of the data recovery bandwidth prediction is further guaranteed.

[0119] In an optional embodiment of the present specification, the above steps of determining the data recovery pressure level based on the comparison result include the following steps:

[0120] If the comparison result shows that the garbage data ratio indicator is less than the third pressure threshold, determining the data recycling pressure level to be the first data recycling pressure level;

[0121] If the comparison result shows that the garbage data ratio indicator is greater than or equal to the third pressure threshold and less than the fourth pressure threshold, determining the data recovery pressure level to be a second data recovery pressure level, wherein the second data recovery pressure level is higher than the first data recovery pressure level;

[0122] When the comparison result shows that the garbage data proportion index is greater than or equal to the fourth pressure threshold, the data recovery pressure level is determined to be the third data recovery pressure level, wherein the third data recovery pressure level is higher than the second data recovery pressure level.

[0123] When the comparison result is that the garbage data ratio index is less than the third pressure threshold, the implementation method of determining the data recovery pressure level as the first data recovery pressure level may be that when the difference between the garbage data ratio index and the third pressure threshold is less than 0, the data recovery pressure level is determined to be the first data recovery pressure level.

[0124] When the comparison result is that the garbage data proportion index is greater than or equal to the third pressure threshold and less than the fourth pressure threshold, the implementation method of determining the data recovery pressure level as the second data recovery pressure level may be that when the difference between the garbage data proportion index and the third pressure threshold is greater than or equal to 0 and the difference with the fourth pressure threshold is less than 0, the data recovery pressure level is determined to be the second data recovery pressure level.

[0125] When the comparison result is that the garbage data ratio index is greater than or equal to the fourth pressure threshold, the implementation method of determining the data recovery pressure level as the third data recovery pressure level may be that when the difference between the garbage data ratio index and the fourth pressure threshold is greater than or equal to 0, the data recovery pressure level is determined to be the third data recovery pressure level.

[0126] By comparing the different results, the data recovery pressure levels are divided into different levels, and the comparison results of different garbage data proportion indicators and thresholds corresponding to different data recovery pressure levels are achieved, which improves the refined division of data recovery pressure levels.

[0127] Step 410: Predicting a data reclamation bandwidth of a storage unit based on the data reclamation pressure level.

[0128] Data recycling bandwidth refers to the bandwidth used when recycling garbage data. Data recycling bandwidth and read and write throughput share the server's bandwidth capacity, and the two are in a competitive relationship. Therefore, it is necessary to set a reasonable bandwidth for data recycling bandwidth to minimize the impact on front-end read and write throughput.

[0129] There are many ways to predict the data recycling bandwidth of a storage unit based on the data recycling pressure level, which are determined according to actual conditions and are not limited in this specification.

[0130] In a possible implementation of the present specification, the data recovery bandwidth of the storage unit is predicted based on the data recovery pressure level. The predicted range of the data recovery bandwidth can be determined based on the storage pressure level; and the data recovery bandwidth of the storage unit can be determined from the predicted range based on the data recovery pressure level.

[0131] In another possible implementation of the present specification, the storage pressure level may be obtained, the storage pressure level and the data recovery pressure level may be input into a bandwidth prediction model, and the data recovery bandwidth output by the bandwidth prediction model may be obtained, wherein the bandwidth prediction model is trained based on the sample storage pressure level, the sample data recovery pressure level and the data recovery bandwidth label.

[0132] In an optional embodiment of the present specification, the above steps of predicting the data recycling bandwidth of the storage unit based on the data recycling pressure level include the following steps:

[0133] Determine a predicted range of data recycling bandwidth according to the storage pressure level, wherein the higher the storage pressure level, the larger the data recycling bandwidth corresponding to the predicted range;

[0134] Based on the data eviction pressure level, a data eviction bandwidth of the storage unit is determined from a prediction range.

[0135] The prediction range is obtained by predicting the set range of the data recycling bandwidth, and the bandwidth value in the prediction range meets the requirements of the storage pressure level.

[0136] The method for determining the predicted range of the data recovery bandwidth according to the storage pressure level may be to predict the executable bandwidth range of the storage unit for data recovery according to the storage pressure level, and determine the predicted executable bandwidth range as the predicted range.

[0137] When the storage pressure level is higher, in order to release the space of the storage unit, the efficiency requirement for garbage data recovery is higher than that of the low storage pressure level, so the corresponding data recovery bandwidth is larger.

[0138] An implementation method for determining the data recovery bandwidth of a storage unit from a prediction range based on the data recovery pressure level may be to select an executable data recovery bandwidth corresponding to the data recovery pressure level from the prediction range based on the data recovery pressure level; or it may be to input the data recovery pressure level and the prediction range into a bandwidth prediction model to obtain the data recovery pressure level output by the bandwidth prediction model, wherein the bandwidth prediction model is obtained based on training of a sample set.

[0139] The predicted range of the data recovery bandwidth is determined by the storage pressure level. Based on the data recovery pressure level, the data recovery bandwidth of the storage unit is determined from the predicted range. When determining the pressure level, the data recovery pressure level is determined on the basis of determining the storage pressure level. When predicting the bandwidth, the predicted range is first determined according to the storage pressure level, and then the predicted data recovery bandwidth is determined from the predicted range according to the data recovery pressure level for data recovery, thereby improving the efficiency of data recovery and the front-end read and write performance.

[0140] In an optional embodiment of the present specification, when data is recycled according to the current data recycling bandwidth in the current time window, a state indicator of a storage unit in the current time window is obtained, and after a data recycling bandwidth is predicted based on the state indicator of the storage unit in the current time window, the data recycling bandwidth of the next time window is guided based on the predicted data recycling bandwidth, so that data recycling is performed according to a reasonable data recycling bandwidth in the next time window. That is, the above step of obtaining the state indicator of the storage unit includes the following steps:

[0141] Get the status indicators of the storage unit in the current time window;

[0142] After the above step of predicting the data recycling bandwidth of the storage unit based on the data recycling pressure level, the following step is also included:

[0143] Get the current data recycling bandwidth for data recycling in the current time window;

[0144] Based on the current data recovery bandwidth and the predicted data recovery bandwidth, a bandwidth change strategy for the data recovery bandwidth is set;

[0145] According to the bandwidth change strategy, the reclaim bandwidth for reclaiming data from the storage unit in the next time window is adjusted, wherein the time interval between the next time window and the current time window is the time interval for predicting the data reclaim bandwidth.

[0146] Specifically, time is divided into multiple time windows based on the window interval. The current time window corresponds to the current time. The window interval is typically 16 seconds and can be set based on actual circumstances. The current data retrieval bandwidth is the actual bandwidth for data retrieval in the current time window. The current data retrieval bandwidth can also be the bandwidth predicted in the previous round. The bandwidth change strategy sets a transitional bandwidth change strategy based on the initial bandwidth and target bandwidth to prevent sudden bandwidth increases from causing storage unit failures.

[0147] An implementation method for setting a bandwidth change strategy for the data recovery bandwidth based on the current data recovery bandwidth and the predicted data recovery bandwidth may be to obtain an initial bandwidth change strategy, update the initial bandwidth change strategy based on the current data recovery bandwidth and the predicted data recovery bandwidth, and obtain a bandwidth change strategy, wherein the initial bandwidth change strategy includes fast recovery and slow start, and fast recovery means that when the predicted data recovery bandwidth is lower than the current data recovery bandwidth, the data recovery bandwidth in the next time window is directly reduced to the predicted data recovery bandwidth, and slow start means that when the predicted data recovery bandwidth is higher than the current data recovery bandwidth, the preset bandwidth threshold is increased at preset time intervals.

[0148] The implementation method of adjusting the recovery bandwidth for data recovery of the storage unit in the next time window according to the bandwidth change strategy can be to determine the change value and change interval corresponding to the next time window according to the bandwidth change strategy, and adjust the recovery bandwidth for data recovery in the next time window based on the change value and change interval.

[0149] For example, the current data recovery bandwidth is 12MB / second, the predicted data recovery bandwidth is 2MB / second, the bandwidth change strategy is to directly reduce it to 2MB / second in the next window, and the change value corresponding to the next time window is determined to be a reduction of 10MB / second, and the change interval is 0 seconds. Based on the change value and the change interval, data recovery is performed at 12MB / second in the next time window.

[0150] By obtaining the status indicators of the storage unit in the current time window, the data recovery bandwidth is predicted based on the status indicators to obtain the predicted data recovery bandwidth, so that data recovery in the next time window can be performed using the predicted data recovery bandwidth. When guiding data recovery in adjacent windows, in order to avoid the impact of sudden increases on the storage unit, it is necessary to set a bandwidth change strategy to achieve smooth bandwidth transition and ensure the normal operation of storage unit data recovery.

[0151] In an optional embodiment of the present specification, the above steps of setting a bandwidth change strategy for the data recycling bandwidth based on the current data recycling bandwidth and the predicted data recycling bandwidth include the following steps:

[0152] When the current data recycling bandwidth is less than the predicted data recycling bandwidth, setting a bandwidth change rate, and gradually adjusting the current data recycling bandwidth to the predicted data recycling bandwidth according to the bandwidth change rate;

[0153] When the current data recycling bandwidth is greater than or equal to the data recycling bandwidth, the data recycling bandwidth of the next time window is set as the predicted data recycling bandwidth.

[0154] When the current data recovery bandwidth is less than the data recovery bandwidth, the bandwidth change rate is set, and the current data recovery bandwidth is gradually adjusted to the predicted data recovery bandwidth according to the bandwidth change rate. The implementation method can be that when the difference between the current data recovery bandwidth and the data recovery bandwidth is less than 0, the bandwidth change rate is set based on the difference, and the current data recovery bandwidth is gradually adjusted to the predicted data recovery bandwidth according to the bandwidth change rate.

[0155] An implementation method for setting the bandwidth change rate based on the difference may be to determine the order of magnitude based on the difference and determine the bandwidth change rate corresponding to the order of magnitude. For example, if the difference is 50MB / s, the order of magnitude to which the difference belongs is determined to be the second order, and the bandwidth change rate corresponding to the second order is determined to be 10MB / s every 5 minutes. For another example, if the difference is 10MB / s, the order of magnitude to which the difference belongs is determined to be the first order, and the bandwidth change rate corresponding to the first order is determined to be 2MB / s every 2 minutes.

[0156] When the current data recovery bandwidth is greater than or equal to the predicted data recovery bandwidth, the data recovery bandwidth of the next time window is set to the predicted data recovery bandwidth. When the difference between the current data recovery bandwidth and the data recovery bandwidth is greater than or equal to 0, the data recovery bandwidth of the next time window is set to the predicted data recovery bandwidth, so as to achieve fast recovery and minimize the impact on read and write performance.

[0157] The comparison results are used to guide the data recovery bandwidth for the next time window. Different comparison results provide different guidance, thereby achieving refined guidance on the data recovery bandwidth for the next time window.

[0158] In an optional embodiment of the present specification, the storage unit is the smallest processing unit for data recovery processing in the storage disk, the storage disk is managed by a stand-alone storage engine, and the stand-alone storage engine is deployed on a storage server.

[0159] A storage server is a hardware device whose core function is to achieve a specific goal, including providing data storage services. Usually, a stand-alone storage engine is deployed on a storage server. The stand-alone storage engine is responsible for managing multiple storage disks. The storage disks are usually SMR disks. The minimum allocated and erased storage unit on the SMR disk is the storage unit in the embodiments of this specification. The storage unit can be a zone.

[0160] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two-dimensional utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and read and write performance.

[0161] Optionally, based on the implementation of one or more embodiments of this specification, accurate prediction of data recovery bandwidth can be further achieved. One method involves using a single factor to drive bandwidth throttling strategies in each system state. For example, a dynamic throttling threshold can be set based on zone utilization or other factors to achieve better adaptability. This can be achieved by combining storage pressure, data recovery pressure, and calculating a dynamic throttling threshold based on zone utilization to predict data recovery bandwidth.

[0162] Among them, another further implementation method is divided into three steps. The first step is: monitor the consumption rate of zones in the system, consumption rate = (number of zones consumed by foreground user write pressure - number of zones released by data recovery) / statistical window time length; the second step is: estimate the number of zones that need to be recovered in the next window, number of zones that need to be recovered = zone consumption rate * statistical window time length; the third step is: estimate the data recovery flow limit threshold based on the number of zones that need to be recovered and the proportion of garbage data in the recovery target zone, garbage data proportion = garbage data size in the zone / total capacity of the zone. For example, in the next window period (assuming 1s), 2 zones (each zone size is 256M) need to be recovered, and the garbage proportion in the target recovery zone is 50%. Then the flow limit threshold is calculated as: flow limit threshold = (256M*2*50%) / 2.

[0163] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two-dimensional utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and read and write performance.

[0164] The following further describes the data recovery method provided in this specification using the application of the data recovery method in a stand-alone storage engine as an example, with reference to FIG5 , wherein FIG5 shows a process flow chart of a data recovery method provided in one embodiment of this specification.

[0165] Start, update the garbage collection current limit threshold, judge the water level pressure, judge the recycling pressure, predict the recycling bandwidth of garbage data, smoothing strategy: slow start and fast recovery, and set the garbage collection current limit.

[0166] When there is a need to update the data recovery bandwidth, the data recovery process will be started, that is, the garbage collection current limit threshold will be updated; the status indicators of the zone in the current time window are obtained from the stand-alone storage engine instance. Among them, the status indicators include the zone utilization indicator, the space utilization indicator, and the garbage data ratio indicator. The zone utilization indicator includes the ratio of the number of closed zones to the total number of zones. The space utilization indicator is the amount of valid data / total disk space (total zone space). The garbage data ratio indicator is the ratio of invalid data in the zone to the total space. For calculation, the zone utilization indicator can be subtracted from the space utilization indicator.

[0167] The water level pressure judgment specifically includes judging the storage pressure level based on the utilization index in the Zone status index: determining the storage pressure level based on the Zone utilization index and the space utilization index, wherein the first storage pressure level is the high water level, and the second storage pressure level is the normal water level.

[0168] Recycling pressure judgment includes obtaining a water level flow control strategy based on the storage pressure level, and then judging the recycling pressure based on the water level flow control strategy and the garbage data ratio indicator to determine the data recycling pressure level.

[0169] When the storage pressure level is the first storage pressure level (high water level), the data recovery pressure level is determined based on the high water level flow limiting strategy and the garbage data ratio indicator to obtain the data recovery pressure level. If the garbage data ratio is less than 2%, the recovery pressure can be determined to be low pressure; if 2% <= garbage data ratio < 3%, the recovery pressure can be determined to be medium pressure; if the garbage data ratio is greater than or equal to 3%, the recovery pressure can be determined to be high pressure;

[0170] When the storage pressure level is the second storage pressure level (normal water level), the data recovery pressure level is judged based on the normal water level flow limiting strategy and the garbage data ratio indicator to obtain the data recovery pressure level. If the garbage data ratio is <2%, the recovery pressure can be determined to be low pressure; if 2% <= the garbage data ratio is <5%, the recovery pressure can be determined to be medium pressure; if the garbage data ratio is >= 5%, the recovery pressure can be determined to be high pressure.

[0171] Predicting the garbage data recovery bandwidth specifically includes predicting the data recovery bandwidth of the storage unit based on the determined data recovery pressure level;

[0172] When the storage pressure level is the first storage pressure level (high water level), the data recovery bandwidth of the storage unit is predicted based on the data recovery pressure level. Specifically, on the basis of the storage pressure level being the first storage pressure level (high water level), if the data recovery pressure level is the first data recovery pressure level (low pressure), the corresponding data recovery bandwidth is 8MB / second; if the data recovery pressure level is the second data recovery pressure level (medium pressure), the corresponding data recovery bandwidth is 20MB / second; if the data recovery pressure level is the third data recovery pressure level (high pressure), the corresponding data recovery bandwidth is 50MB / second.

[0173] When the storage pressure level is the second storage pressure level (normal water level), the data recovery bandwidth of the storage unit is predicted based on the data recovery pressure level. Specifically, on the basis of the storage pressure level being the second storage pressure level (normal water level), if the data recovery pressure level is the first data recovery pressure level (low pressure), the corresponding data recovery bandwidth is 2MB / second; if the data recovery pressure level is the second data recovery pressure level (medium pressure), the corresponding data recovery bandwidth is 12MB / second; if the data recovery pressure level is the third data recovery pressure level (high pressure), the corresponding data recovery bandwidth is 30MB / second.

[0174] Smoothing strategies: slow start and fast recovery. Slow start: The current data recovery bandwidth in the current time window is 2 MB / s, and the predicted data recovery bandwidth is 12 MB / s. That is, when the data recovery bandwidth jumps and increases, it increases by 2 MB / s every two minutes. Fast recovery: The current data recovery bandwidth in the current time window is 12 MB / s, and the predicted data recovery bandwidth is 2 MB / s. No smoothing is required, and the data recovery bandwidth is directly reduced to 2 MB / s in the next time window to quickly reduce the impact of the data recovery bandwidth on read and write performance.

[0175] Garbage collection current limiting settings include guiding data collection in the next time window based on a smoothing strategy.

[0176] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two dimensions of the utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and the read and write performance.

[0177] Referring to FIG. 6 , FIG. 6 shows a schematic diagram of system module interaction in a data recovery method provided in an embodiment of the present disclosure.

[0178] The system module includes a system status collection and analysis module and a data recovery and throttling module. The system status collection and analysis module includes both collection and analysis functions. The collection function periodically collects system status indicators, while the analysis function updates the data recovery and throttling threshold based on the indicators. The data recovery and throttling threshold represents the data recovery bandwidth. The data recovery and throttling module limits the data recovery process based on the updated data recovery and throttling threshold.

[0179] A storage engine instance is deployed in the storage server. The storage engine instance is responsible for managing multiple storage disks, which include multiple minimum allocation and erase unit zones. The system status collection and analysis module periodically collects status indicators of the storage units in the storage engine instance, where the periodicity can be a window interval. Based on the collected status indicators, the system status collection and analysis module updates the data recovery flow limiting threshold, that is, updates the data recovery bandwidth, and sends the data recovery bandwidth to the data recovery flow limiting module. When the data recovery flow limiting module receives the driver flow limiting update, it limits the bandwidth of the data recovery process based on the updated data recovery flow limiting threshold.

[0180] The solution of one or more embodiments of this specification enables a stand-alone storage engine based on SMR HDD to have self-analysis capabilities by adding a system key indicator collection and analysis module.

[0181] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two-dimensional utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and read and write performance.

[0182] In one or more embodiments of this specification, the data recovery method is tested, specifically including the following situations:

[0183] The first scenario: When the storage pressure is above 90% and all the data stored in the Zone are valid data, and there is no need to trigger data recovery, the read and write throughput of a single machine is doubled, specifically from 1.22GB / s to 2.6GB / s. Please refer to Figures 7a and 7b below for details. Figure 7a shows a schematic diagram of the throughput of a single machine under high storage pressure and no data recovery scenario in a data recovery method provided by an embodiment of this specification, wherein the horizontal axis is time, and the time includes "20:50, 21:00, 21:10, 21:20, 21:20, 21:30, 21:40 and 21:50", the vertical axis is bandwidth, the vertical axis unit is MB / s, and the bandwidth includes "0, 500, 1000, 1500, 2000, 2500, 3000", and Figure 7b shows A schematic diagram of data recovery in a data recovery method provided in an embodiment of this specification under high storage pressure and no data recovery scenario is shown, wherein the horizontal axis is the time axis, and the times include "20:50, 21:00, 21:10, 21:20, 21:20, 21:30, 21:40 and 21:50"; the vertical axis is the recovery bandwidth of garbage data, the unit of the vertical axis is GB / second, and the recovery bandwidth includes "0, 2, ..., 12". At this time, because the data recovery pressure is not high, the data recovery bandwidth is maintained at the lowest speed of 8MB / second.

[0184] Under the same test conditions, when the data recovery bandwidth is set based on a single factor, high storage pressure and low garbage data ratio still produce a very high data recovery bandwidth. Please refer to Figures 7c and 7d for details. Figure 7c shows a schematic diagram of the single-machine throughput under high storage pressure and no data recovery scenario in a current data recovery method, where the horizontal axis is the time axis, and the time includes "14:00, 14:10, ..., 15:00", and the vertical axis is "0, 1C, 2C, 3C", where C is 10MB / s; Figure 7d shows a schematic diagram of the single-machine throughput under high storage pressure and no data recovery scenario in a current data recovery method. Schematic diagram of data recovery in a scenario without data recovery. The horizontal axis is the time axis, including "14:00, 14:10, ..., 15:00", and the vertical axis is the data recovery bandwidth, in MB / s, including "0, 25, 50, 75". In the 85% high storage pressure and high garbage data scenario, the background data recovery bandwidth is actively improved. With a garbage data ratio of 6%, the single-machine throughput remains at around 2GB / s. In the 97% extreme high storage pressure and random deletion scenario, a write throughput capacity of 20MB / s per disk and 1.2GB / s per machine is provided.

[0185] The second scenario: Under normal storage pressure and a high proportion of garbage data, the background data recovery bandwidth is actively improved. When the garbage data proportion is 6%, the single-machine throughput remains at around 2GB / s.

[0186] Referring to Figures 7e and 7f, Figure 7e shows a schematic diagram of the throughput of a single machine under normal storage pressure and high garbage data scenarios in a data recovery method provided by an embodiment of this specification, wherein the horizontal axis is the time axis, and the time includes "14:00, 14:10, ..., 15:00", the vertical axis is "0, 1C, 2C, 3C", C is 10MB / s, and the dotted line is the bandwidth change of write throughput; Figure 7f shows a schematic diagram of the throughput of a single machine under normal storage pressure and high garbage data scenarios in another data recovery method provided by an embodiment of this specification, wherein the horizontal axis and the vertical axis are the same as those in Figure 7e. As can be seen in Figures 7e and 7f, one or more embodiments of this specification accurately judge that in the early stage of the test, the system is at a normal water level, a low recovery pressure state, and the data recovery bandwidth is set to a low level; as the test progresses, the proportion of garbage data gradually increases, and the system state successively experiences a medium recovery pressure state and a high recovery pressure state, and the data recovery flow limiting bandwidth also changes adaptively accordingly. It is achieved that stable throughput performance can be provided under different system states.

[0187] In data recovery current limiting smoothing: slow start and fast recovery, avoiding performance degradation caused by sudden increase, see Figure 7g, which shows a schematic diagram of the change of data recovery bandwidth in a data recovery method provided by an embodiment of this specification, wherein the horizontal axis is the time axis, and the time includes "14:00, 14:10, ..., 15:00", the vertical axis is "0, 1C, 2C, 3C", C is 10MB / second, the dotted line is the recovery bandwidth of garbage collection, and the content in the solid box is the focus of attention. In Figure 7g, it takes about 10 minutes to increase from 12MB / second to about 30MB / second at a speed of 2MB / min.

[0188] The scheme of one or more embodiments of this specification introduces four key system indicators as reference factors for data recovery flow control, and defines different system states from the two dimensions of water level pressure and recovery pressure; through the accurate mobile phone system state, reasonable data recovery flow control thresholds are set in a targeted manner, ensuring that the single-machine storage engine provides stable input / output (I / O) performance, and also setting a slow start and fast recovery flow control smoothing strategy to avoid sudden impact on foreground input / output (I / O) performance.

[0189] Corresponding to the above method embodiment, this specification also provides a data recovery device embodiment. FIG8 shows a schematic diagram of the structure of a data recovery device provided in one embodiment of this specification. As shown in FIG8, the device includes:

[0190] A first acquisition module 802 is configured to acquire a status indicator of a storage unit, wherein the status indicator includes a usage indicator and a garbage data ratio indicator of the storage unit;

[0191] A first determination module 804 is configured to determine a storage pressure level according to a usage rate indicator;

[0192] A second acquisition module 806 is configured to acquire a data recovery pressure level classification strategy based on the storage pressure level, wherein the data recovery pressure level classification strategy includes a classification parameter for determining the data recovery pressure level, and the higher the storage pressure level, the higher the data recovery pressure level determined by the classification parameter;

[0193] The second determining module 808 is configured to determine the data recycling pressure level according to the garbage data ratio indicator and the data recycling pressure level classification strategy;

[0194] The prediction module 810 is configured to predict the data recycling bandwidth of the storage unit based on the data recycling pressure level.

[0195] Optionally, the number of storage units is at least one, and the utilization index includes a storage unit utilization index and a space utilization index. The storage unit utilization index is the proportion of saturated storage units in at least one storage unit, and a saturated storage unit is a storage unit to which data cannot be written but is readable. The space utilization index is the proportion of valid data in at least one storage unit in the total data storage space. The first determination module 804 is further configured to compare the storage unit utilization index and the space utilization index with the corresponding pressure thresholds based on a storage pressure level classification strategy, wherein the storage pressure level classification strategy refers to a strategy for classifying storage units into storage pressure levels. The storage pressure level is determined based on the comparison result.

[0196] Optionally, the first determination module 804 is further configured to determine that the storage pressure level is the first storage pressure level when the comparison result is that the storage unit utilization index is greater than or equal to the first pressure threshold, or the space utilization index is greater than or equal to the second pressure threshold; and to determine that the storage pressure level is the second storage pressure level when the comparison result is that the storage unit utilization index is less than the first pressure threshold and the space utilization index is less than the second pressure threshold, and the second storage pressure level is lower than the first storage pressure level.

[0197] Optionally, the second determination module 808 is further configured to compare the garbage data ratio index with the level range threshold according to the data recovery pressure level classification strategy, wherein the garbage data ratio index is the proportion of invalid data in the storage unit in the total data storage space, and the level range threshold is composed of a third pressure threshold and a fourth pressure threshold, and the third pressure threshold is less than the fourth pressure threshold; based on the comparison result, the data recovery pressure level is determined.

[0198] Optionally, the second determination module 808 is further configured to determine the data recovery pressure level as the first data recovery pressure level when the comparison result is that the garbage data proportion index is less than the third pressure threshold; determine the data recovery pressure level as the second data recovery pressure level when the comparison result is that the garbage data proportion index is greater than or equal to the third pressure threshold and less than the fourth pressure threshold, wherein the second data recovery pressure level is higher than the first data recovery pressure level; and determine the data recovery pressure level as the third data recovery pressure level when the comparison result is that the garbage data proportion index is greater than or equal to the fourth pressure threshold, wherein the third data recovery pressure level is higher than the second data recovery pressure level.

[0199] Optionally, the prediction module 810 is further configured to determine a prediction range of the data recovery bandwidth based on the storage pressure level, wherein the higher the storage pressure level, the larger the data recovery bandwidth corresponding to the prediction range; based on the data recovery pressure level, the data recovery bandwidth of the storage unit is determined from the prediction range.

[0200] Optionally, the first acquisition module 802 is further configured to obtain a status indicator of the storage unit in the current time window; the data recovery device also includes a setting module, configured to obtain the current data recovery bandwidth for data recovery in the current time window; based on the current data recovery bandwidth and the predicted data recovery bandwidth, set a bandwidth change strategy for the data recovery bandwidth; according to the bandwidth change strategy, adjust the recovery bandwidth for data recovery of the storage unit in the next time window, wherein the time interval between the next time window and the current time window is the time interval for predicting the data recovery bandwidth.

[0201] Optionally, the setting module is further configured to set the bandwidth change rate when the current data recovery bandwidth is less than the predicted data recovery bandwidth, and gradually adjust the current data recovery bandwidth to the predicted data recovery bandwidth according to the bandwidth change rate; when the current data recovery bandwidth is greater than or equal to the data recovery bandwidth, set the data recovery bandwidth of the next time window to the predicted data recovery bandwidth.

[0202] Optionally, the storage unit is the smallest processing unit for data recovery processing in the storage disk, and the storage disk is managed by a stand-alone storage engine, which is deployed on a storage server.

[0203] By applying the solution of the embodiments of this specification, storage pressure levels are divided according to the utilization index in the status index of the storage unit, and a data recovery pressure division strategy corresponding to the storage pressure is obtained based on the storage pressure levels obtained by the division. When the data recovery pressure levels are divided based on the data recovery pressure division strategy, the division is performed on the basis of the storage pressure levels, that is, the division is performed based on the two-dimensional utilization index and the garbage data ratio index, so that the utilization rate of the storage unit and the garbage data ratio are taken into consideration when determining the data recovery bandwidth, thereby ensuring the rationality of the data recovery bandwidth setting, and thereby significantly improving the data recovery performance and read and write performance.

[0204] The above is a schematic diagram of a data recovery device according to this embodiment. It should be noted that the technical solution of the data recovery device and the technical solution of the above-mentioned data recovery method are of the same concept. For details not described in detail in the technical solution of the data recovery device, please refer to the description of the technical solution of the above-mentioned data recovery method.

[0205] Figure 9 shows a block diagram of a computing device according to one embodiment of the present disclosure. Components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0206] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0207] In one embodiment of the present specification, the aforementioned components of computing device 900 and other components not shown in FIG9 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG9 is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0208] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 900 may also be a mobile or stationary server.

[0209] The processor 920 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data recovery method.

[0210] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned data recovery method are of the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned data recovery method.

[0211] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which implement the steps of the above-mentioned data recovery method when executed by a processor.

[0212] The above is a schematic diagram of a computer-readable storage medium of this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above-mentioned data recovery method are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-mentioned data recovery method.

[0213] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data recovery method.

[0214] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above-mentioned data recovery method are of the same concept. For details not described in detail in the technical scheme of the computer program, please refer to the description of the technical scheme of the above-mentioned data recovery method.

[0215] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0216] Computer instructions include computer program code, which may be in source code form, object code form, executable files, or some intermediate form. Computer-readable media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content of computer-readable media may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunications signals.

[0217] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0218] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0219] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to specific embodiments. Obviously, many modifications and variations are possible based on the content of the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data recovery method, comprising: Acquire a status indicator of the storage unit, wherein the status indicator includes a usage rate indicator and a junk data ratio indicator of the storage unit; Determining a storage pressure level according to the utilization rate indicator; Based on the storage pressure level, acquiring a data recovery pressure level classification strategy, wherein the data recovery pressure level classification strategy includes a level classification parameter for determining the data recovery pressure level, and in the case where the storage pressure level is higher, the data recovery pressure level determined by the level classification parameter is higher; Determine the data recovery pressure level according to the garbage data proportion index and the data recovery pressure level classification strategy; A data reclamation bandwidth of the storage unit is predicted based on the data reclamation pressure level.

2. The method according to claim 1, wherein the number of the storage unit is at least one, the usage index comprises a storage unit usage index and a space usage index, the storage unit usage index is the proportion of saturated storage units in at least one storage unit in the at least one storage unit, the saturated storage unit is a storage unit to which data cannot be written and is readable, and the space usage index is the proportion of valid data in at least one storage unit in the total data storage space; Determining the storage pressure level according to the usage rate indicator includes: Based on a storage pressure level classification strategy, the storage unit utilization index and the space utilization index are respectively compared with corresponding pressure thresholds, wherein the storage pressure level classification strategy refers to a strategy for classifying storage pressure levels of the storage units; Based on the comparison results, the storage pressure level is determined.

3. The method according to claim 2, wherein determining the storage pressure level according to the comparison result comprises: When the comparison result is that the storage unit usage index is greater than or equal to the first pressure threshold, or the space usage index is greater than or equal to the second pressure threshold, determining the storage pressure level to be the first storage pressure level; When the comparison result is that the storage unit utilization index is less than the first pressure threshold and the space utilization index is less than the second pressure threshold, the storage pressure level is determined to be the second storage pressure level, which is lower than the first storage pressure level.

4. According to the method of claim 1, determining the data recovery pressure level according to the garbage data proportion index and the data recovery pressure level classification strategy comprises: According to the data recycling pressure level division strategy, the garbage data proportion index is compared with the level range threshold, wherein the garbage data proportion index is the proportion of invalid data in the storage unit in the total data storage space, and the level range threshold consists of a third pressure threshold and a fourth pressure threshold, and the third pressure threshold is less than the fourth pressure threshold; Based on the comparison results, the data recovery pressure level is determined.

5. The method according to claim 4, wherein determining the data recovery pressure level based on the comparison result comprises: When the comparison result shows that the garbage data proportion index is less than the third pressure threshold, determining the data recovery pressure level to be the first data recovery pressure level; If the comparison result is that the garbage data proportion index is greater than or equal to the third pressure threshold and less than the fourth pressure threshold, determining the data recovery pressure level to be the second data recovery pressure level, wherein the second data recovery pressure level is higher than the first data recovery pressure level; When the comparison result is that the garbage data proportion index is greater than or equal to the fourth pressure threshold, the data recovery pressure level is determined to be a third data recovery pressure level, wherein the third data recovery pressure level is higher than the second data recovery pressure level.

6. The method according to claim 1, wherein the step of predicting the data recycling bandwidth of the storage unit based on the data recycling pressure level comprises: Determining a predicted range of a data recycling bandwidth according to the storage pressure level, wherein, in a case where the storage pressure level is higher, the data recycling bandwidth corresponding to the predicted range is larger; Based on the data reclamation pressure level, a data reclamation bandwidth of the storage unit is determined from the predicted range.

7. The method according to claim 1, wherein obtaining the status indicator of the storage unit comprises: Get the status indicators of the storage unit in the current time window; After predicting the data recycling bandwidth of the storage unit based on the data recycling pressure level, the method further includes: Get the current data recycling bandwidth for data recycling in the current time window; Based on the current data recovery bandwidth and the predicted data recovery bandwidth, setting a bandwidth change strategy for the data recovery bandwidth; According to the bandwidth change strategy, the recovery bandwidth for data recovery of the storage unit in the next time window is adjusted, wherein the time interval between the next time window and the current time window is the time interval for predicting the data recovery bandwidth.

8. The method according to claim 7, wherein the step of setting a bandwidth change strategy for the data recycling bandwidth based on the current data recycling bandwidth and the predicted data recycling bandwidth comprises: In a case where the current data recycling bandwidth is less than the predicted data recycling bandwidth, setting a bandwidth change rate, and gradually adjusting the current data recycling bandwidth to the predicted data recycling bandwidth according to the bandwidth change rate; When the current data recycling bandwidth is greater than or equal to the predicted data recycling bandwidth, the data recycling bandwidth of the next time window is set to be the predicted data recycling bandwidth.

9. According to the method of claim 1, the storage unit is the smallest processing unit for data recovery processing in the storage disk, the storage disk is managed by a stand-alone storage engine, and the stand-alone storage engine is deployed on a storage server.

10. A data recovery device, comprising: A first acquisition module is configured to acquire a status indicator of a storage unit, wherein the status indicator includes a usage rate indicator and a junk data ratio indicator of the storage unit; A first determination module is configured to determine a storage pressure level according to the usage rate indicator; A second acquisition module is configured to acquire a data recovery pressure level division strategy based on the storage pressure level, wherein the data recovery pressure level division strategy includes a level division parameter for determining the data recovery pressure level, and in the case where the storage pressure level is higher, the data recovery pressure level determined by the level division parameter is higher; A second determination module is configured to determine the data recovery pressure level according to the garbage data proportion index and the data recovery pressure level classification strategy; The prediction module is configured to predict the data recycling bandwidth of the storage unit based on the data recycling pressure level.

11. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the steps of the method according to any one of claims 1 to 9.

13. A computer program, when executed in a computer, causes the computer to execute the steps of the method according to any one of claims 1 to 9.

Images