Verification method and system for space release operation, electronic equipment and chip

Abstract

The invention discloses a space release operation verification method and system, electronic equipment and a chip. The method comprises the steps that a space release request of a simulation pressure gauge is monitored, and the space release request is obtained through construction based on a junk data index corresponding to a junk storage space under the condition that the simulation pressure gauge monitors that the junk storage space exists in the storage space of a storage server; the junk data index is used for searching junk data in the to-be-processed data file associated with the load in the junk storage space; calling an operation interface to respond to the space release request, executing a space release operation on the junk storage space based on the junk data index, and releasing junk data in the junk storage space; in the process of releasing the junk data in the junk storage space, obtaining a storage index of the storage space; and verifying the storage index to obtain a verification result. The technical problem that the verification cost of the space release operation is high is solved.

Description

Technical Field

[0001] This application relates to the field of computers, and in particular, to a method, system, electronic device, and chip for verifying a space release operation. Background Art

[0002] Currently, Trim File is a new interface and capability provided by a distributed storage system based on a storage server to provide a fine-grained space resource release capability inside a file for a storage server scenario. For example, Trim File is an interface and capability provided for a Zoned Namespace Solid State Disk (ZNS SSD). A space release operation can be performed on the garbage storage space in the storage server through Trim File. Therefore, it is very necessary to test and verify the Trim File technology to ensure the effect of the storage service.

[0003] In the related art, when evaluating the Trim File technology, a full-link evaluation method for a storage server scenario is usually adopted. This method generally includes steps such as cluster construction, server deployment, verification scenario construction, and full-link parameter and configuration adjustment. The above method not only has cumbersome evaluation steps and complex coordination, but also highly depends on hardware and modules for parameter tuning, resulting in a high evaluation cost. Therefore, there is still a technical problem of high verification cost for space release operations.

[0004] In response to the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] Embodiments of this application provide a method, system, electronic device, and chip for verifying a space release operation to at least solve the technical problem of high verification cost for space release operations.

[0006] According to one aspect of the embodiments of the present application, a method for verifying a space release operation is provided. The method is applied to a stand-alone storage engine deployed on a storage server, and a simulation pressure device is also deployed on the storage server. The simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located. The method may include: monitoring the space release requests of the simulation pressure device, where the space release requests are constructed based on the garbage data indexes corresponding to the garbage storage spaces when the simulation pressure device detects that there are garbage storage spaces in the storage space of the storage server. The garbage data indexes are used to find the garbage data in the data files to be processed associated with the load in the garbage storage spaces; calling an operation interface to respond to the space release requests, and based on the garbage data indexes, performing a space release operation on the garbage storage spaces to release the garbage data in the garbage storage spaces; during the process of releasing the garbage data in the garbage storage spaces, obtaining the storage metrics of the storage space; verifying the storage metrics to obtain a verification result.

[0007] According to another aspect of the embodiments of the present application, another method for verifying a space release operation is provided. The method is applied to a simulation pressure device deployed on a storage server. The simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located, and a stand-alone storage engine is also deployed on the storage server. The method may include: monitoring the storage space of the storage server; when it is detected that there is a garbage storage space in the storage space, obtaining the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data files to be processed associated with the load in the garbage storage space; constructing a space release request based on the garbage data index; sending the space release request to the operation interface of the stand-alone storage engine; where the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0008] According to another aspect of the embodiments of the present application, another method for verifying a space release operation is provided. The method is applied to a storage server and may include: monitoring the storage space of the storage server; when it is detected that there is a garbage storage space in the storage space, obtaining the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data files to be processed associated with the load in the garbage storage space; constructing a space release request based on the garbage data index; calling an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage spaces, obtaining the storage metrics of the storage space; verifying the storage metrics to obtain a verification result.

[0009] According to another aspect of the embodiments of the present application, a verification system for space release operations is further provided. The system may include: a simulation pressure device for monitoring the storage space; when it is detected that there is a garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the data files associated with the load in the garbage storage space; constructing a space release request based on the garbage data index; a single-machine storage engine for calling an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtaining storage metrics of the storage space; and verifying the storage metrics to obtain a verification result.

[0010] According to another aspect of the embodiments of the present application, an electronic device is further provided. The electronic device may include a memory and a processor: the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the above computer-executable instructions are executed by the processor, the verification method for space release operations in the embodiments of the present application is implemented.

[0011] According to another aspect of the embodiments of the present application, a processor is further provided. The processor is used to run a program, and when the program is running, the verification method for space release operations in the embodiments of the present application is executed.

[0012] According to another aspect of the embodiments of the present application, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, and when the program is running, it controls the device where the storage medium is located to execute the verification method for space release operations in the above embodiments of the present application.

[0013] According to another aspect of the embodiments of the present application, a chip is further provided. The chip includes a processor configured to call and run a computer program from a memory, so that the device installed with the chip executes the verification method for space release operations in the above embodiments of the present application.

[0014] In the embodiments of the present application, during the process of simulating the load in the storage service scenario where the storage server is located through a simulation pressure device, the storage space in the storage server can be monitored in real time through the simulation pressure device. When it is detected that there is garbage storage space in the storage space, garbage data in the to-be-processed data files associated with the load can be found in the garbage storage space, and a garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated according to the garbage data index. In response to the space release request, the corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained, and the performance of writing data to the storage space can be evaluated through the storage metrics, and the storage metrics are verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiments of the present application consider that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependencies and shortening the verification link of the effect of this technology is achieved. Furthermore, the technical effect of reducing the verification cost of the space release operation is realized, and the technical problem of high verification cost of the space release operation is solved.

[0015] It is easy to note that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation to the present application. Brief Description of the Drawings

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0017] Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a verification method for space release operation according to an embodiment of the present application;

[0018] Figure 2 is a structure block diagram of a computing environment for a verification method for space release operation according to an embodiment of the present application;

[0019] Figure 3 is a flowchart of a verification method for space release operation according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of another verification method for space release operation according to an embodiment of the present application;

[0021] Figure 5 is a flowchart of another verification method for space release operation according to an embodiment of the present application;

[0022] Figure 6 It is a flowchart of a verification system for a space release operation according to an embodiment of the present application;

[0023] Figure 7 It is a schematic diagram of the deployment situation of a Pangu file according to an embodiment of the present application;

[0024] Figure 8 It is a schematic diagram of a Pangu TrimFile capability according to an embodiment of the present application;

[0025] Figure 9 It is a schematic diagram of a cluster for evaluating Trim effects in a related technology according to an embodiment of the present application;

[0026] Figure 10 It is a schematic diagram of another cluster for evaluating Trim effects in a related technology according to an embodiment of the present application;

[0027] Figure 11 It is a schematic diagram of the evaluation results of a TrimFile technology in a full-link test according to an embodiment of the present application;

[0028] Figure 12 It is a schematic diagram of an evaluation system for a Trim technology according to an embodiment of the present application;

[0029] Figure 13 It is a schematic diagram of a functional fuzzer module according to an embodiment of the present application;

[0030] Figure 14 It is a schematic diagram of a garbage collection method for a Trim technology according to an embodiment of the present application;

[0031] Figure 15 It is a schematic diagram of the verification results of a Trim effect according to an embodiment of the present application;

[0032] Figure 16 It is a schematic diagram of the comparison of Trim write amplification for closing and opening according to an embodiment of the present application;

[0033] Figure 17 It is a schematic diagram of a verification device for a space release operation according to an embodiment of the present application;

[0034] Figure 18 It is a schematic diagram of another verification device for a space release operation according to an embodiment of the present application;

[0035] Figure 19 It is a schematic diagram of another verification device for a space release operation according to an embodiment of the present application;

[0036] Figure 20 It is a block diagram of a computer terminal according to an embodiment of the present application;

[0037] Figure 21 It is a block diagram of an electronic device for a verification method of a space release operation according to an embodiment of the present application. Detailed implementation manners

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0039] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0040] First, some nouns or terms that appear during the description of the embodiments of the present application are applicable to the following explanations:

[0041] A distributed storage system refers to storing data on multiple independent storage servers and managing services and data through a network. The multi-copy mechanism provides high reliability, high availability, and high scalability. Common distributed storage systems can include highly fault-tolerant systems (Hadoop Distributed File System, abbreviated as HDFS), Amazon Simple Storage Service (abbreviated as Amazon 3S) systems, and distributed file systems (Ceph), etc.;

[0042] A storage cluster. The distributed storage system links multiple storage servers into a storage cluster. A typical storage cluster can include thousands of storage servers;

[0043] A single-machine storage engine is a service process running on each storage server, used to provide storage services on a single storage server. It can include functions such as high-performance reading and writing, user input / output (IO) request scheduling, data integrity scanning, and abnormal data reporting. In the Pangu distributed storage system, the single-machine storage engine refers to the ChunkServer. In the embodiments of this application, the single-machine storage engine refers to the single-machine storage engine customized based on ZNS SSD;

[0044] Pangu file. Pangu provides a file semantics of the write link (append-only) for the functional module, that is, it only supports append writing and does not support overwriting the written positions. In fact, usually a Pangu file is divided into variable-length data blocks, that is, chunks. Each chunk is saved to different ChunkServers in the form of multiple replicas or encoding;

[0045] TrimFile capability and the Trim operation of the single-machine storage engine. Usually, a Pangu file is a small unit of resource management, and the space can only be released when the entire file is deleted. The TrimFile capability can specify a certain range of data in the file for deletion, and the garbage storage space can be released without deleting the entire file. The actual space release operation can be executed by the single-machine storage engine where the corresponding replica is located, that is, by calling the Trime operation of the ZNS SSD single-machine storage engine;

[0046] Elastic Block Storage (EBS) service provides cloud disks for users;

[0047] Cloud disk garbage data. Since EBS is based on the Log-Structured Merge Tree (LSM Tree) architecture, new data is written into the Pangu file in the form of append writing. When using the cloud disk, it is part of the data in the Pangu file;

[0048] EBS garbage collection. The EBS cloud disk data corresponds to multiple Pangu files. The valid data is read from the Pangu file and written into a new Pangu file, and then the old Pangu file is deleted, so as to achieve the behavior of deleting garbage data and releasing storage space, which is called EBS garbage collection.

[0049] Embodiment 1

[0050] According to an embodiment of the present application, a method for verifying a space release operation is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0051] The method embodiment provided in Embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal, or a similar computing device. Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for verifying a space release operation according to an embodiment of the present application, as Figure 1 shown, the computer terminal 10 (or mobile device) may include one or more (shown as 102a, 102b,..., 102n in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microcontroller unit (abbreviated as MCU) or a field programmable gate array (abbreviated as FPGA)), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition to this, it may further include: a display, an input / output interface (I / O interface), a universal serial bus (abbreviated as USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0052] Figure 1 The shown hardware structure block diagram can not only be used as an exemplary block diagram of the above computer terminal 10 (or mobile device), but also as an exemplary block diagram of the above server. In an alternative embodiment, Figure 2 is shown in a block diagram an embodiment using the above Figure 1 shown computer terminal 10 (or mobile device) as a computing node in a computing environment 201.

[0053] Figure 2 is a structure block diagram of a computing environment for a method for verifying a space release operation according to an embodiment of the present application, as Figure 2As shown, the computing environment 201 includes multiple computing nodes (such as servers, shown in the figure as 210-1, 210-2, …) running on a distributed network. Each computing node contains local processing and memory resources, and end users 202 can remotely run applications or store data in the computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3, and 220-4 in the computing environment 201, representing services “A”, “D”, “E”, and “H” respectively.

[0054] End users 202 can provide and access services through a web browser or other software applications on the client side. In some embodiments, the provision and / or requests of end users 202 can be provided to the ingress gateway 230. The ingress gateway 230 can include a corresponding proxy to handle the provision and / or requests for services (one or more services provided in the computing environment 201).

[0055] Services are provided or deployed according to various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar means. VM-based virtualization can be achieved by initializing virtual machines to simulate real computers and executing programs and applications without directly accessing any actual hardware resources. While virtualizing machines with virtual machines, according to container-based virtualization, containers can be launched to virtualize the entire operating system so that multiple workloads can run on a single operating system instance.

[0056] In one embodiment of container-based virtualization, several containers of a service can be assembled into a Pod (for example, a Kubernetes Pod). For example, as Figure 2 shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, …, 240-N (collectively referred to as Pods). A Pod can include a proxy 245 and one or more containers 242-1, 242-2, …, 242-M (collectively referred to as containers). One or more containers in a Pod handle requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with Pods similar to this.

[0057] During operation, executing user requests from end users 202 may require invoking one or more services in the computing environment 201, and executing one or more functions of a service may require invoking one or more functions of another service. As Figure 2As shown, service "A" 220-1 receives a user request from end-user 202 at the ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0058] The computing environment described above may be a cloud computing environment, where the allocation of resources is managed by a cloud service provider, allowing the development of functions without considering the implementation, adjustment, or expansion of servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically scaled independently, rather than scaling a single hardware device to handle potential loads.

[0059] In the above operating environment, the present application provides a verification method for the space release operation as shown in Figure 3 This method can be deployed in a single-machine storage engine on a storage server. It should be noted that the verification method for the space release operation in this embodiment can be executed by Figure 1 the mobile terminal shown in the embodiment. Figure 3 is a flowchart of a verification method for a space release operation according to an embodiment of the present application. As shown in Figure 3 this method may include the following steps:

[0060] Step S302, monitor the space release request of the simulation pressure device. Among them, the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server. The garbage data index is used to find the garbage data in the data file associated with the load in the garbage storage space.

[0061] In the technical solution provided in step S302 of the present application, the space release request of the simulation pressure device can be monitored. Among them, the storage server can be called a cluster and can be a distributed storage system. For example, it can be the Pangu distributed file system, which can include ZNS SSD. The single-machine storage engine can be the single-machine storage engine of Pangu. For example, it can be the Pangu Chunk Server. A simulation pressure device can also be deployed on the storage server. The simulation pressure device can be used to at least simulate the load in the storage service scenario where the storage server is located. It can be a functional simulation pressure device. For example, it can be a single-machine pressure device, in which a functional pressure model can be deployed. The storage service scenario can be the upper-layer function, the usage scenario of the LSM Tree architecture, or the EBS functional module. The load can be called the load characteristic. For example, it can be the IO load characteristic. The simulation pressure device can include a valid data index and a garbage data index. The garbage data index can be represented as the identifier (Key) of the garbage data contained in the chunk table. The chunk table can also include a valid data index. The storage space is the storage space of the storage device in the storage server. The storage engine data can include valid data and garbage data. The space release request can be constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server. The space release request can be a Trim request. The garbage storage space can be the Trim garbage space or can also be called the invalid space. The garbage data index can be used to find the garbage data in the to-be-processed data file associated with the load in the garbage storage space and can also be called the invalid data index. The to-be-processed data file can be a Pangu file.

[0062] Optionally, the load characteristics in the storage service scenario are at least simulated by the simulation pressure device on the storage server. And during the simulation process, the simulation pressure device can also perform real-time detection on the storage space of the memory server. When garbage data in the to-be-processed data file related to the load is found in the storage space, a corresponding space release request can be constructed according to the corresponding garbage data index.

[0063] Optionally, according to the IO load analysis of the EBS functional module, the IO load characteristics of the EBS can be simulated in a single-machine environment by the single-machine pressure device. During the simulation process, it can include simulating the data written to the front end of the cloud disk and the garbage collection flow. The simulation of processes such as the Trim operation of the single-machine storage engine accessed by the newly added process can also be included. It should be noted that the processes simulated by the simulation pressure device above are only for illustrative purposes and are not specifically limited here.

[0064] In the related art, the process of verifying TrimFile through steps such as cluster setup, server deployment, verification scenario construction, and full-link parameter and configuration adjustment is not only cumbersome but also involves complex coordination. Usually, it includes setting up 10 clusters for verification. Therefore, there are still technical problems of low verification efficiency and high cost for the space release operation. However, in the embodiments of the present application, the evaluation dependence of TrimFile is simplified. A single-machine pressure device is used to simulate the load situation of the storage service scenario in a single-machine environment (single storage node). During the simulation process, the process of the space release operation can be verified, thus greatly simplifying the evaluation process's dependence on multiple components such as clusters, EBS services, and Pangu services. As a result, only one ZNS SSD server is needed to run the evaluation test process of the space release operation triggered by TrimFile, achieving the technical effects of not only simplifying the evaluation process but also reducing the number of servers used, and further improving the verification efficiency of the space release operation and reducing costs.

[0065] Step S304: Call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0066] In the technical solution provided in step S304 of the present application, after detecting the space release request of the simulation pressure device, the operation interface can be called to respond to the space release request. Based on the garbage data index, a space release operation is performed on the garbage storage space to release the corresponding garbage data in the garbage storage space. Among them, the operation interface can be a new interface, for example, the Trime operation interface.

[0067] Optionally, after detecting that the simulation pressure device has a space release request, the garbage data can be screened out from the data files to be processed corresponding to the garbage data index in the space release request by calling the operation interface according to the garbage data index included in the space release request, and the garbage data can be released from its garbage storage space.

[0068] For example, in this embodiment for TrimFile technology, the core implementation is the Trim operation of the single-machine storage engine, which realizes the deletion function for a segment of data. That is, after detecting the space release request, the operation interface can be called, and according to the garbage data index in the space release request, the corresponding garbage data is determined for deletion, so as to release the garbage storage space storing the garbage data.

[0069] Step S306: During the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space.

[0070] In the technical solution provided in step S306 of the present application, when the operation interface is called to respond to the space release request, based on the garbage data index, a release operation is performed on the garbage storage space to release the garbage data in the garbage storage space. During the process of releasing the garbage data in the garbage storage space, the storage metrics in the storage space can be obtained. The storage metrics can be used to represent the performance of writing data to the storage space. By determining the performance of writing data, the garbage collection traffic after the TrimFile function is started in the embodiments of the present application is evaluated. For example, the storage metrics can be write amplification information. The data written to the storage space can be summarized, calculated, analyzed, etc. through the write amplification information to perform amplification processing on the data, so as to better display the trend and characteristics of the data, and determine the performance of writing data to the storage space, thereby evaluating whether the garbage collection traffic is reduced after starting TrimFile. In the embodiments of the present application, the write amplification information can also be referred to as (function-level) write amplification. For example, the write amplification in the EBS function scenario (EBS write amplification).

[0071] It should be noted that the above storage metrics being write amplification information is only for illustrative purposes and is not specifically limited here. As long as the storage metrics can determine the performance of writing data to the storage space and thereby evaluate whether the garbage collection traffic decreases, they are within the protection scope of the embodiments of the present application.

[0072] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called, and the data obtained during the current release process is compared with the business layer write amplification to verify the function and expected application effect of the Trim technology.

[0073] Step S308: Verify the storage metrics to obtain a verification result.

[0074] In the technical solution provided in step S308 of the present application, during the process of releasing the garbage data in the garbage storage space, after obtaining the storage metrics of the storage space, the storage metrics can be verified to obtain a verification result. The verification result can be used to represent the effect of the space release operation, that is, it can be used to represent the function and expected application effect of the Trim technology.

[0075] Optionally, verify the storage metrics during the process of releasing the garbage data to determine whether the function of the Trim technology can achieve the expected application effect.

[0076] For example, under the same configuration, the business layer write amplification during the process of opening Trim to release garbage data can be collected, and the business layer write amplification when Trim is closed can also be collected in advance. Through the comparison of the above two business layer write amplifications, the verification result of whether the Trim technology can achieve the expected effect is determined.

[0077] It should be noted that the above processes and methods for analyzing whether the Trim technology meets the expected effects are only for illustrative purposes and are not specifically limited here.

[0078] In the embodiment of the present application, by using the simulation pressure device in the storage server, the load in the storage service scenario can be simulated in a single-machine environment, and the Trim technology of the ZNS SSD storage engine can be applied in the simulation pressure device. By comparing the write methods at the service layer, the functions and expected application effects of the Trim technology can be verified, thereby improving the verification efficiency of the space release operation. And in this process, the evaluation dependencies are streamlined. Using the simulation pressure device in a single-machine environment, based on the functional pressure model, the triggering of garbage collection is reduced, and the write amplification at the service layer is reduced, thereby achieving the technical effect of reducing the verification cost of the space release operation.

[0079] Through the above steps S302 to S308 of the present application, during the process of simulating the load in the storage service scenario where the storage server is located by using the simulation pressure device, the storage space in the storage server can be monitored in real time through the simulation pressure device. When it is detected that there is garbage storage space in the storage space, the garbage data in the data files to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated according to the garbage data index. In response to the space release request, the corresponding operation interface is called, and based on the garbage data index, the space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained, and the performance of writing data to the storage space can be evaluated through the storage metrics, and the storage metrics are verified to obtain the verification result. The functions and service effects of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present application considers that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependencies and shortening the verification link of the technical effects is achieved, and further the technical effect of reducing the verification cost of the space release operation is realized, and the technical problem of high verification cost of the space release operation is solved.

[0080] The above method of this embodiment will be further introduced below.

[0081] As an alternative implementation, in step S304, the operation interface is called to respond to the space release request. Based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data in the garbage storage space, including: calling the operation interface to respond to the space release request, and in the garbage storage space, searching for the initial storage location where the garbage data is located based on the garbage data index; and deleting the garbage data at the initial storage location in the garbage storage space.

[0082] In this embodiment, during the process of calling the operation interface to respond to the space release request, performing a space release operation on the garbage storage space based on the garbage data index, and releasing the garbage data in the garbage storage space, the operation interface can be called to respond to the space release request. In the garbage storage space, the initial storage location where the garbage data is located is searched based on the garbage data index, and the garbage data is deleted at the initial storage location in the garbage storage space. Herein, the initial location can also be referred to as the old location or the specified location.

[0083] Optionally, after detecting the space release request, the operation interface can be called to analyze the space release request, analyze the garbage data index corresponding to the garbage data for which the space release operation is to be performed from the space release request, search for the initial storage location where the garbage data is located through the garbage data index, and delete the garbage data at the initial storage location.

[0084] In the related art, by triggering the garbage collection process, the valid data in a Chunk is read from the initial location, all written to a new location, and the Chunk at the old location is deleted, so as to release the storage space occupied by the garbage. However, the embodiment of the present application is different from the above-mentioned garbage collection. The Trim technology can be used to delete the garbage data at the specified location in the Chunk to release the garbage storage space here, and avoid data recovery reading and writing, thereby achieving the purpose of reducing the garbage collection traffic and reducing the write amplification, and further solving the technical problem of high verification cost for the space release operation.

[0085] As an alternative implementation, the storage space includes a valid storage space in addition to the garbage storage space. Herein, deleting the garbage data at the initial storage location in the garbage storage space includes: determining the data block in the garbage storage space that includes the garbage data; identifying the valid data in the data file to be processed in the data block; reading the valid data from the data block; and when the read valid data is successfully written to the valid storage space, deleting the data block including the garbage data at the initial storage location to delete the garbage data.

[0086] In this embodiment, the storage space may further include valid storage space other than the garbage storage space. At the initial position in the garbage storage space, during the process of deleting garbage data, the data blocks where the garbage data is located in the garbage storage space can be determined. The valid data in the data file to be processed can be identified in the data block, the valid data can be read out from the data block, and the read valid data can be processed and written to the corresponding target storage position in the valid storage space. After the valid data is successfully written to the target storage position, the data block including the garbage data at the initial storage position can be deleted to delete the garbage data. Among them, the data block including the garbage data at the initial storage position can be an old Chunk. The valid storage space can be available space.

[0087] Optionally, through a function simulation pressure device, in a single storage node environment, based on a function pressure model, the foreground write traffic, garbage collection read and write traffic, and Trim traffic are simultaneously simulated.

[0088] Optionally, the storage engine data file may include multiple Chunks. A Chunk only supports append writing, and its characteristics are the same as those of the data file in the EBS scenario. A Chunk may contain valid data and garbage data in the garbage storage space.

[0089] Optionally, a Chunk may contain multiple data with a unique identifier Key. By marking (Put) the Key for the data (Value), such as garbage data and valid data, and writing it to the Chunk in append mode, the ChunkTable in the storage space and the Location Table in the local can be updated.

[0090] Optionally, during the process of performing an update or delete process, if an update or delete operation is performed on the Key, then the ChunkTable in the storage space needs to be updated, and the Key needs to be updated or deleted from the valid data index and the LocationTable, and the corresponding data space is added to the invalid data index.

[0091] Optionally, after a space release request is detected, it can be determined whether to trigger the Trim process. If so, the garbage data index of the ChunkTable can be scanned to obtain garbage space information. The Trim operation of the single-machine storage engine can be called to release the garbage storage space. After the garbage storage space is successfully released, new available Chunks can be created as needed to support more writes.

[0092] As an alternative implementation, during the process of releasing garbage data in the garbage storage space, storage metrics of the storage space are obtained, including: during the process of releasing garbage data in the garbage storage space, detecting a first amount of written data of valid data written to the valid storage space; and determining the storage metrics based on the first amount of written data.

[0093] In this embodiment, during the process of releasing garbage data in the garbage storage space, the first amount of written data of valid data written to the valid storage space can be detected, and the storage metrics can be determined based on the first amount of written data. Herein, the first amount of written data can be the amount of data written during garbage collection. The target storage location can be used to represent a new file to which valid data is written.

[0094] Optionally, when a key is deleted or updated, it indicates that a garbage storage space is generated. The proportion of the garbage storage space in the current system can be evaluated by statistically analyzing and updating the garbage ratio. By analyzing the current space situation, it can be determined whether the Trim process is triggered. If it is determined that garbage collection is required, the garbage collection process needs to be triggered. For example, garbage collection is required when the free chunks in the FreeChunkList are insufficient. And the first amount of written data can be determined to update the storage metrics, that is, the write amplification information can be updated. This write amplification information can be statistically analyzed periodically and does not necessarily depend on the energy recovery process.

[0095] Optionally, when the Trim condition is met, the simulator can scan the garbage index data in the ChunkTable to construct a Trim request.

[0096] Optionally, the Trim operation interface of the storage engine can be called to send the Trim request to the storage engine.

[0097] Optionally, during the process of releasing the garbage data corresponding to the space release request, by calling the interface for obtaining storage information of the stand-alone storage epidemic, it can be confirmed whether the garbage storage space is successfully released. After the space is successfully released, new writes can be supported according to the remaining space and the remaining available chunks.

[0098] Optionally, during the process of releasing garbage data, the amount of data that reads valid data and writes it to a new file can be read, so as to obtain the first amount of written data.

[0099] For example, the first amount of written data can be statistically analyzed through the Garbage Collection Worker (abbreviated as GC Worker) in EBS.

[0100] As an alternative implementation, the method further includes: during the process of releasing the garbage data in the garbage storage space, detecting a second amount of written data obtained by performing a write operation on the storage space; determining a storage metric based on the first amount of written data, including: determining the storage metric based on the first amount of written data and the second amount of written data.

[0101] In this embodiment, during the process of releasing the garbage data in the garbage storage space, it is also possible to detect the second amount of written data obtained by performing a write operation on the storage space, and the storage metric can be determined through the first amount of written data and the second amount of written data. Among them, the second amount of written data can be the amount of data written to the cloud disk.

[0102] Optionally, during the process of releasing the garbage data, it is possible to detect the actual amount of data written by the user in the cloud disk to obtain the second amount of written data.

[0103] For example, the second amount of written data can be counted through the BlockServer module in the EBS.

[0104] Optionally, after determining the first amount of written data and the second amount of written data, the storage metric can be calculated through the first amount of written data and the second amount of written data.

[0105] For example, after obtaining the second amount of written data through the above method, the EBS write amplification can be determined according to the following formula, that is, EBS write amplification = (amount of data written to the cloud disk + amount of data written during garbage collection) / amount of data written to the cloud disk.

[0106] As an alternative implementation, in step S308, verifying the storage metric to obtain a verification result includes: in response to a shutdown instruction, obtaining an initial storage metric of the storage space in the storage service scenario, where the shutdown instruction can be used to indicate the prohibition of calling the operation interface; verifying the storage metric using the initial storage metric to obtain a verification result.

[0107] In this embodiment, during the process of verifying the storage metric to obtain a verification result, based on the shutdown instruction, the operation interface is prohibited from being called, and in this case, the initial storage metric of the storage space in the storage service scenario can be obtained, and then the storage metric is verified using the initial storage metric to obtain a verification result. Among them, the shutdown instruction can be used to indicate the prohibition of calling the operation interface. The operation interface can be the operation interface corresponding to the TrimFl ie function. The initial storage metric can be used to represent the write amplification in the case of turning off the TrimFi le function.

[0108] Optionally, when TrimFile is closed, initial storage metrics can be collected from the garbage data released from the garbage storage space during the garbage collection process at this time. The initial storage metrics can be the write amplification detected when the TrimFile function of the distributed storage system is closed.

[0109] Optionally, when TrimFile is open, storage metrics can be collected from the garbage data released from the garbage storage space during the garbage collection process at this time. The above storage metrics can be the write amplification detected when the TrimFile function of the distributed storage system is open.

[0110] Optionally, based on the above two storage metrics, it can be determined whether the TrimFile function can achieve the expected effect.

[0111] For example, if the write amplification detected when the TrimFile function of the distributed storage system is open is greater than the write amplification detected when the TrimFile function of the distributed storage system is closed, it can be shown that starting the TrimFile function has a certain effect on reducing the garbage collection write volume.

[0112] For example, it can be confirmed by comparing the two cases of using and not using the TrimFile function whether the write amplification in the case of using TrimFile is less than that in the case of closing TrimFile. If so, it can be shown that the TrimFile function has achieved the expected effect. Otherwise, it can be shown that the TrimFile function has not achieved the expected effect. It should be noted that this is only an example, and there is no specific limitation on the process and method of determining whether the TrimFile function has achieved the expected effect.

[0113] As an optional implementation, the method further includes: when the garbage data in the garbage storage space is successfully released, based on the released garbage storage space, updating the available storage space in the storage space; and invoking an information interface to perform a write operation on the updated available storage space.

[0114] In this embodiment, when the garbage data in the garbage storage space is successfully released, the available storage space in the storage space can be updated based on the released garbage storage space, and an information interface can be invoked to perform a write operation on the updated available storage space to write data. The information interface can be an interface for obtaining storage information in a single-machine storage engine.

[0115] Optionally, it is possible to detect whether the garbage data has been successfully released. When it is detected that the garbage data has been successfully released based on the space release request, the garbage storage space after the garbage data has been released can be updated. That is, the garbage storage space that has released garbage data can be updated to an effective storage space for writing subsequent data that needs to be written.

[0116] Optionally, through the interface for obtaining storage information in the single-machine storage engine, it is possible to detect whether the garbage data has been successfully released. After confirming that the garbage data has been successfully released, the garbage storage space can be adjusted to an effective storage space. New writes can be performed according to the remaining space and the remaining available Chunks.

[0117] The embodiment of the present application also provides a verification method for the space release operation from the simulated pressure side. Figure 4 It is a flowchart of a verification method for the space release operation according to the embodiment of the present application, as Figure 4 shown. This method is applied to a simulated pressure device deployed on a storage server. The simulated pressure device is used to simulate the load in the storage service scenario where the storage server is located. A single-machine storage engine is also deployed on the storage server. The method may include the following steps:

[0118] Step S402, monitor the storage space of the storage server.

[0119] In the technical solution provided in step S402 of the present application above, the storage space in the storage server can be monitored through the simulated pressure device.

[0120] Optionally, at least the load characteristics in the storage service scenario are simulated through the simulated pressure device on the storage server. During the simulation process, the storage space of the memory server can also be detected in real time through the simulated pressure device.

[0121] Optionally, according to the IO load analysis of the EBS function, the IO load characteristics of the EBS can be simulated in a single-machine environment through a single-machine pressure device. During the simulation process, it may include simulating the writes and garbage collection flows in the front end of the cloud disk. It is also possible to add a process to access the simulation of operations such as the Trim operation of the single-machine storage engine.

[0122] In the embodiments of the present application, the evaluation dependence of TrimFile is simplified. A single-machine pressure device is used to simulate the load situation of the storage service scenario in a single-machine environment. During the simulation process, the process of the space release operation can be verified, thus greatly simplifying the evaluation process's multiple dependencies on clusters, EBS services, Pangu services, etc. Therefore, only one ZNS SSD server is needed to run the evaluation test process of the space release operation triggered by TrimFile, achieving the technical effects of not only simplifying the evaluation process but also reducing the servers used, and further realizing the improvement of the verification efficiency of the space release operation and the reduction of costs.

[0123] Step S404, when it is monitored that there is garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data file to be processed associated with the load in the garbage storage space.

[0124] In the technical solution provided in step S404 of the present application above, when it is monitored that there is garbage storage space in the storage space, the garbage data index corresponding to the garbage storage space can be obtained, where the garbage data index can be used to find the garbage data in the data file to be processed associated with the load in the garbage storage space.

[0125] Optionally, when the garbage data in the data file to be processed related to the load is found in the storage space, a corresponding space release request can be constructed according to the corresponding garbage data index.

[0126] When the garbage data in the data file to be processed related to the load is found in the storage space, a corresponding space release request can be constructed according to the corresponding garbage data index.

[0127] Step S406, construct a space release request based on the garbage data index.

[0128] In the technical solution provided in step S406 of the present application above, after it is monitored that there is garbage storage space in the storage space and thus the garbage data index corresponding to the garbage storage space is obtained, a space release request can be constructed based on the garbage data index.

[0129] Optionally, a space release request is constructed based on the garbage data index, that is, the garbage data index corresponding to the garbage data to be released can be stored in the space release request.

[0130] Optionally, it is monitored whether there is garbage storage space in the storage space. In the case where garbage storage space is detected, the garbage data index corresponding to the garbage storage space can be obtained from the chunk table. The garbage index data can include the chunk identifier (Chunkid) in the chunk table and the location where the corresponding garbage data is located (List <location>)。After obtaining the above Chunkid and List<Location>, they can be filled in the space release request. Thus, the space release request can be sent to the single-machine storage engine to prompt the single-machine storage engine with the locations of the garbage data to be trimmed.

[0131] It should be noted that the garbage data index used to determine the garbage data to be modified in the above construction of the space release request is only for illustrative purposes and is not specifically limited here. As long as it can reflect the location of the garbage data and indicate the single-machine storage engine to trim it, the space release request is within the protection scope of the embodiments of the present application.

[0132] Step S408: Send the space release request to the operation interface of the single-machine storage engine. Among them, the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0133] In the technical solution provided in step S408 of the present application above, after constructing the space release request based on the garbage data index, the space release request can be sent to the operation interface of the single-machine storage engine. Among them, the space release request can be responded to by the operation interface, and the garbage data index in the space release request can be used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the corresponding garbage data in the garbage storage space, and during the process of releasing the garbage data in the garbage storage space, the storage metrics of the storage space are verified to obtain a verification result.

[0134] Optionally, the space release request can be sent to the operation interface through a simulation pressure device. The corresponding garbage data can be released through the operation interface, and the release process can be verified to obtain a verification result.

[0135] Optionally, after detecting that the simulation pressure device has a space release request, according to the garbage data index corresponding to the garbage data to be released included in the space release request, the operation interface can be called to screen out the garbage data from the data files to be processed corresponding to the garbage data index, and the garbage data can be released from its garbage storage space.

[0136] For example, for the TrimFile technology, the core implementation is the Trim operation of the single-machine storage engine, which realizes the deletion function for a segment of data. That is, after detecting the space release request, the operation interface can be called, and according to the garbage data index in the space release request, the corresponding garbage data can be determined for deletion, so as to release the garbage storage space storing the garbage data.

[0137] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, business layer write amplification can be called. By comparing the data obtained during the current release process with the business layer write amplification, the functions of the Trim technology and the expected application effects can be verified.

[0138] Optionally, verify the storage metrics during the process of releasing garbage data to determine whether the functions of the Trim technology can achieve the expected application effects.

[0139] For example, under the same configuration, the business layer write amplification during the process of releasing garbage data with Trim enabled can be collected, or the business layer write amplification with Trim disabled can be collected in advance. By comparing the above two business layer write amplifications, the verification result of whether the Trim technology can achieve the expected effects can be determined.

[0140] Through steps S402 to S408 of the present application above, monitor the storage space of the storage server; in the case where a garbage storage space is detected in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data file associated with the load in the garbage storage space; construct a space release request based on the garbage data index; send the space release request to the operation interface of the single-machine storage engine; where the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0141] The above method of this embodiment will be further introduced below.

[0142] As an optional implementation manner, the method may include: in the case where there is a data identifier of deletion or update in the simulation pressure device, determine that there is a garbage storage space in the storage space, where the data identifier is used to represent the data stored in the storage space.

[0143] In this embodiment, in the case where there is a data identifier of deletion or update in the simulation pressure device, the existing garbage storage space in the storage space can be determined, where the data identifier can be used to identify the data stored in the storage space. The data identifier can be a Key and can be the unique identifier of the data stored in the storage space.

[0144] Optionally, during the process of monitoring the storage space in the storage server through a simulation pressure device, it is possible to monitor whether there are data identifiers to be deleted or updated in the storage server through the simulation pressure device. When it is detected that there are data identifiers to be deleted or updated, it is possible to determine whether there is any garbage storage space in the storage space.

[0145] Optionally, when it is detected through the simulation pressure device that a key is deleted or updated in the storage server, it can be explained that garbage space is generated at this time. Then, it is necessary to release the garbage data in the garbage space by constructing a corresponding space release request.

[0146] As an alternative implementation, step S404, obtaining the garbage data index corresponding to the garbage storage space, includes: obtaining the proportion of the garbage storage space in the storage space; in response to the proportion meeting the proportion threshold, obtaining the garbage data index corresponding to the garbage storage space.

[0147] In this embodiment, during the process of obtaining the garbage data index corresponding to the garbage storage space, it is possible to determine whether the garbage storage space meets the target condition. The target condition may be whether the proportion of the garbage storage space in the storage space meets the proportion threshold. If it does not meet, the garbage data index corresponding to the garbage storage space can be obtained. Among them, the target condition can be used to indicate whether the garbage storage space reaches the trigger Trim process. The proportion can also be called the garbage ratio (Stales). The proportion threshold can be a value set in advance according to the size of the storage space of the storage device in the storage server, or a value set by itself according to the actual demand for space release. For example, this value can be set to 5% or 15% in advance. It should be noted that the size and setting method of the above proportion threshold are only for illustrative purposes and are not specifically limited here.

[0148] Optionally, by analyzing the situation of the storage space in the current storage server, it is possible to determine whether the trigger Trim process is reached.

[0149] For example, if the target condition is to determine whether the FreeChunk in the storage space is sufficient and the FreeChunk is insufficient, it can be explained that the garbage storage space meets the target condition. At this time, the garbage collection process can be triggered, that is, the garbage data index can be obtained to construct a space release request.

[0150] As an alternative implementation, the method further includes: in response to the proportion not meeting the proportion threshold, performing a garbage collection operation on the storage space.

[0151] In this embodiment, the proportion of the garbage storage space in the storage space can be obtained. When it is determined that the proportion does not meet the proportion threshold, it is necessary to perform a garbage collection operation on the storage space.

[0152] Optionally, each time a garbage collection operation is performed, that is, each time a Key is deleted or updated, it is necessary to use a simulated stressor to count and update the garbage ratio, that is, the proportion of garbage storage space to storage space, to evaluate the proportion of garbage space in the current system. When the proportion is greater than or equal to the proportion threshold, the garbage ratio cannot meet the proportion threshold. This means that there is a lot of garbage data in the storage space, and it is necessary to determine the garbage data index corresponding to the garbage data to be pruned, and trigger the corresponding garbage collection operation through the garbage data index.

[0153] Optionally, it is also possible to determine whether it is necessary to start a garbage collection operation by determining the proportion of effective storage space to storage space. When it is determined that the proportion does not meet a certain proportion threshold, a garbage collection operation can be performed on the storage space. For example, the proportion threshold of effective storage space to storage space can be pre-set to 95%. After each deletion or update of a Key, the proportion of effective storage space to the storage space of the entire storage device can be obtained in real time through a simulation pressure device. When the proportion is greater than or equal to the proportion threshold, it can be said that the proportion meets the proportion threshold, and garbage collection is not required. When the proportion is less than the proportion threshold, it can be said that the proportion threshold is not met, and garbage collection is required to increase the proportion of effective storage space until it is greater than or equal to 95%, and then garbage collection can be stopped.

[0154] For example, in an embodiment of the present application, it is also possible to determine whether it is necessary to start a garbage collection operation by the ratio between the effective storage space and the garbage storage space. The target condition can be pre-set as whether the ratio between the effective storage space and the garbage storage space is greater than or equal to the ratio threshold (19:1). After each deletion or update of a Key, the ratio between the effective storage space and the garbage storage space can be obtained in real time through a simulation pressure device. When the ratio is greater than 19:1, it can be said that the garbage storage space meets the target condition, and garbage collection is not required at this time. When the ratio is less than 19:1, it can be said that the garbage storage space does not meet the target condition. At this time, it is necessary to determine the garbage data index of the garbage data for which the garbage collection operation is to be performed, and construct a corresponding space release request to trim the garbage data in the storage space to recycle the garbage storage space.

[0155] It should be noted that the above-mentioned target conditions for determining whether a garbage collection operation needs to be performed are only for illustration and are not specifically limited here. As long as the target conditions can trigger the execution of a garbage collection operation, they are within the protection scope of the embodiments of the present application.

[0156] As an optional implementation, the method further includes: in response to the garbage storage space not satisfying the target condition, performing a garbage collection operation on the storage space.

[0157] In this embodiment, when the garbage storage space does not meet the target condition, a garbage collection operation can be performed on the storage space.

[0158] Optionally, if it is determined that garbage collection is required, the garbage collection process needs to be triggered.

[0159] As an alternative embodiment, the storage space includes an effective storage space in addition to the garbage storage space. The method further includes: when the valid data in the data file to be processed is successfully written from the garbage storage space to the effective storage space, updating the memory index data result of the analog pressure device, where the memory index data structure includes a garbage data index and a valid data index for valid data, and the valid data index is used to find valid data in the effective storage space.

[0160] In this embodiment, the storage space may include an effective storage space. When the valid data in the data file to be processed can be successfully written from the garbage storage space to the effective storage space, the memory index data result of the analog pressure device can be updated. The memory index data structure may include a garbage data index and a valid data index for valid data, and the valid data index can be used to find valid data in the effective storage space. The memory index data structure may be a memory data structure, such as ChunkTable.

[0161] Optionally, the memory data interface may include a valid data index and an invalid data index. The valid data index can maintain the Key of the valid data in each Chunk, that is, maintain the unique identifier of the valid data in a data file in each corresponding EBS scenario. The invalid data index can maintain the position of the invalid space in each Chunk. By backtracking through the valid data index, the position of the invalid data can also be obtained.

[0162] In the embodiment of the present application, an invalid data index is added, which can more quickly find the invalid space in a Chunk.

[0163] As an alternative embodiment, the memory index data structure further includes at least one of the following: the position of the garbage data corresponding to the garbage data index in the data block where it is located, the position of the valid data corresponding to the valid data index in the data block where it is located, and the list of valid data blocks where the data block is located.

[0164] In this embodiment, the memory index data structure may further include at least one of the following: the position of the garbage data corresponding to the garbage data index in the data block where it is located, the position of the valid data corresponding to the valid data index in the data block where it is located, and the list of valid data blocks where the data block is located.

[0165] Optionally, the Location Table maintains the position of the data corresponding to a Key in a Chunk, that is, the offset and data length of the Key in that Chunk. The FreeChunkList can maintain a list of available Chunks.

[0166] As an alternative implementation, a simulator is used to store multiple data chunks maintained by a single-machine storage engine.

[0167] In this embodiment, multiple data chunks maintained by a single-machine storage engine can be stored through a simulator.

[0168] In the embodiments of the present application, a storage engine maintains multiple Chunks. The Chunk only supports append writes, and its characteristics are the same as those of the data files in the EBS scenario.

[0169] Optionally, a Chunk can contain valid data and garbage space. And multiple above-mentioned Chunks can be stored in a simulator.

[0170] The embodiments of the present application also provide a method for verifying a space release operation. Figure 5 is a flowchart of a method for verifying a space release operation according to an embodiment of the present application, as Figure 5 shown. This method may include the following steps:

[0171] Step S502, monitor the storage space of the storage server.

[0172] In the technical solution provided in step S502 of the present application above, the storage space of the storage server can be monitored.

[0173] Optionally, during the process of monitoring the storage space of the storage server through a simulator, it is possible to monitor whether there are data identifiers for deletion or update in the storage server through the simulator. When it is detected that there are data identifiers for deletion or update, it is possible to determine whether there is garbage storage space in the storage space.

[0174] Step S504, when it is detected that there is garbage storage space in the storage space, obtain a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the garbage storage space for the data file associated with the load to be processed.

[0175] In the technical solution provided in step S504 of the present application above, when it is detected that there is garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space can be obtained, where the garbage data index can be used to find the garbage data of the data file associated with the load to be processed in the garbage storage space.

[0176] Optionally, by analyzing the storage space situation in the current storage server, it is determined whether to trigger the Trim process.

[0177] For example, if the target condition is to determine whether the FreeChunk in the storage space is sufficient, and the FreeChunk is insufficient, it can be explained that the garbage storage space meets the target condition. At this time, the garbage collection process can be triggered, that is, the garbage data index can be obtained to construct a space release request.

[0178] Step S506, construct a space release request based on the garbage data index.

[0179] In the technical solution provided in step S506 of the present application, a space release request can be constructed based on the garbage data index.

[0180] Optionally, a space release request can be constructed based on the garbage data index, that is, the garbage data index corresponding to the garbage data to be released can be stored in the space release request.

[0181] Step S508, call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0182] In the technical solution provided in step S508 of the present application, after detecting the space release request of the simulation pressure device, the operation interface can be called to respond to the space release request, and based on the garbage data index, a space release operation can be performed on the garbage storage space to release the corresponding garbage data in the garbage storage space.

[0183] Optionally, after detecting that the simulation pressure device has a space release request, the garbage data can be screened out from the data files to be processed corresponding to the garbage data index according to the garbage data index included in the space release request, and the garbage data can be released from its garbage storage space.

[0184] For example, for the TrimFi le technology, the core implementation is the Trim operation of the single-machine storage engine, which realizes the deletion function for a segment of data. That is, after detecting the space release request, the operation interface can be called, and according to the garbage data index in the space release request, the corresponding garbage data can be determined for deletion, so as to release the garbage storage space storing the garbage data.

[0185] Step S510, during the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space.

[0186] In the technical solution provided in step S510 of the present application, when the operation interface is called to respond to the space release request, based on the garbage data index, a release operation is performed on the garbage storage space to release the garbage data in the garbage storage space, and during the process of releasing the garbage data in the garbage storage space, the storage metrics in the storage space can be obtained.

[0187] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the write amplification of the service layer can be called, and the data obtained during the current release process is used to compare with the write amplification of the service layer to verify the function and expected application effect of the Trim technology.

[0188] Step S512: Verify the storage metrics to obtain a verification result.

[0189] In the technical solution provided in step S512 of the present application, during the process of releasing the garbage data in the garbage storage space, after obtaining the storage metrics of the storage space, the storage metrics can be verified to obtain a verification result.

[0190] Optionally, verify the storage metrics during the process of releasing the garbage data to determine whether the function of the Trim technology can achieve the expected application effect.

[0191] For example, under the same configuration, the write amplification of the service layer during the process of opening Trim to release garbage data can be collected, or the write amplification of the service layer when Trim is closed can be collected in advance. By comparing the above two write amplifications of the service layer, the verification result of whether the Trim technology can achieve the expected effect can be determined.

[0192] In the embodiment of the present application, using the simulation press in the storage server can simulate the load in the storage service scenario in a single-machine environment, and the Trim technology of the ZNS SSD storage engine can be applied in the simulation press. By comparing the write methods of the service layer, the function and expected application effect of the Trim technology are verified, thereby improving the verification efficiency of the space release operation. And during this process, the evaluation dependence is streamlined. Using the simulation press in a single-machine environment, based on the functional pressure model, the triggering of garbage collection is reduced, and the write amplification at the service layer is reduced, thus achieving the technical effect of reducing the verification cost of the space release operation.

[0193] Through the above steps S502 to S512 of the present application, the storage space of the storage server is monitored; in the case where a garbage storage space exists in the monitored storage space, the garbage data index corresponding to the garbage storage space is obtained, where the garbage data index is used to find garbage data in the to-be-processed data file associated with the load in the garbage storage space; a space release request is constructed based on the garbage data index; the space release request is sent to the operation interface of the single-machine storage engine; where the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0194] Embodiment 2

[0195] According to an embodiment of the present application, an embodiment of a verification system for a space release operation is further provided. Figure 6 is a schematic diagram of a verification system for a space release operation according to an embodiment of the present application, as Figure 6 shown, the verification system 600 for the space release operation may include: a simulation pressure device 601 and a single-machine storage engine 602.

[0196] The simulation pressure device 601 is configured to monitor the storage space; in the case where a garbage storage space exists in the monitored storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the to-be-processed data file associated with the load in the garbage storage space; construct a space release request based on the garbage data index.

[0197] In this embodiment, the storage space can be monitored by the simulation pressure device 601, and in the case where a garbage storage space exists in the monitored storage space, the garbage data index corresponding to the garbage storage space can be obtained. And a space release request can be constructed based on the garbage data index.

[0198] Optionally, the simulation pressure device 601 on the storage server is used to at least simulate the load characteristics in the storage service scenario, and during the simulation process, the simulation pressure device 601 can also perform real-time detection on the storage space of the memory server. When garbage data in the to-be-processed data file related to the load is found in the monitored storage space, a corresponding space release request can be constructed according to the corresponding garbage data index.

[0199] Optionally, according to the IO load analysis of the EBS function, the IO load characteristics of EBS can be simulated in a single-machine environment through a single-machine press. During the simulation, it can include simulating the write and garbage collection flows on the front end of the cloud disk. It can also add the simulation of processes such as the Trim operation of the single-machine storage engine accessed by the new process.

[0200] Optionally, after the simulation press 601 detects a space release request, the space release request can be sent to the single-machine storage engine 602.

[0201] The single-machine storage engine 602 is used to call the operation interface to respond to the space release request, perform a space release operation on the garbage storage space based on the garbage data index, and release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space; verify the storage metrics to obtain a verification result.

[0202] In this embodiment, the single-machine storage engine 602 can be used to call the operation interface to respond to the space release request, perform a space release operation on the garbage storage space based on the garbage data index, and release the garbage data in the garbage storage space. During the process of releasing the garbage data in the garbage storage space, the storage metrics of the storage space can be obtained, and by verifying the storage metrics, a verification result can be obtained.

[0203] Optionally, after detecting that the simulation press has a space release request, according to the garbage data index corresponding to the garbage data to be released included in the space release request, call the operation interface to screen out the garbage data from the data files to be processed by the garbage data index, and the garbage data can be released from its corresponding garbage storage space.

[0204] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called, and the data obtained during the current release process is compared with the business layer write amplification to verify the function and expected application effect of the Trim technology.

[0205] Optionally, verify the storage metrics during the process of releasing the garbage data to determine whether the function of the Trim technology can achieve the expected application effect.

[0206] For example, under the same configuration, the business layer write amplification during the process of opening Trim to release garbage data can be collected, and the business layer write amplification when Trim is closed can also be collected in advance. By comparing the above two business layer write amplifications, the verification result of whether the Trim technology can achieve the expected effect can be determined.

[0207] In this embodiment, a verification system for space release operation is provided. The storage space is monitored through a simulation pressure device; in the case where garbage storage space exists in the monitored storage space, a garbage data index corresponding to the garbage storage space is obtained, where the garbage data index is used to find garbage data in the to-be-processed data file associated with the load in the garbage storage space; a space release request is constructed based on the garbage data index; the operation interface is called through the stand-alone storage engine to respond to the space release request, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, the storage metrics of the storage space are obtained; the storage metrics are verified to obtain a verification result, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0208] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application, for example, the data to be verified, are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0209] Embodiment 3

[0210] Currently, TrimFile is a new interface and capability provided by the Pangu distributed file system based on the ZNS SSD stand-alone storage engine, providing the upper-layer service with the ability to release fine-grained space resources inside the file. The core technical point is the stand-alone Trim technology provided by the ChunkServer of the Pangu stand-alone storage engine. Compared with the previous whole file (usually several hundred MB to several GB in size) as the smallest unit for space resource management, the Trim-related technology refines the space resource management unit into a certain section of data in the file (typically 256KB in a typical scenario). The business layer of the LSM Tree architecture can release garbage space in a fine-grained manner without frequent garbage collection, so there are significant application advantages.

[0211] In the related art, when evaluating the TrimFile technology, the full-link evaluation method in the storage server scenario is usually adopted. This method usually includes steps such as cluster construction, server deployment, verification scenario construction, and full-link parameter and configuration adjustment. The above method not only has cumbersome evaluation steps and complex coordination, but also highly depends on hardware and modules for parameter tuning, resulting in a high evaluation cost. Therefore, there is still the technical problem of high verification cost for space release operations.

[0212] Furthermore, the present application provides an evaluation method for the Trim technology of the ZNS SSD single-machine storage engine, which solves the technical problem of high verification cost for space release operations. Different from the traditional solution that involves steps such as cluster construction, server deployment, verification scenario construction, and full-link parameter and configuration adjustment, however, the above method not only has cumbersome evaluation steps and complex coordination, but also highly depends on hardware and modules for parameter tuning, resulting in high evaluation costs. It solves the technical problem of high verification cost for space release operations.

[0213] In the embodiment of the present application, during the process of simulating the load in the storage service scenario where the storage server is located through a simulation pressor, the storage space in the storage server can be monitored in real time. When it is detected that there is garbage storage space in the storage space, garbage data in the data files to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated based on the garbage data index. In response to the space release request, the corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained and verified to obtain a verification result. The function and service effect of the Trim File technology can be evaluated through the verification result. Since the embodiment of the present application considers that the evaluation process of the Trim File technology can be simplified through the above steps, the purpose of streamlining the evaluation dependence and shortening the verification link of the technology effect is achieved, and further the technical effect of reducing the verification cost for space release operations is realized, solving the technical problem of high verification cost for space release operations.

[0214] The above method of this embodiment will be further introduced below.

[0215] In this embodiment, Figure 7 is a schematic diagram of the deployment of a Pangu file according to an embodiment of the present application. As Figure 7 shown, the Pangu distributed file system provides the file semantics of the storage engine link for the functional modules, that is, it only supports append writes and does not support overwriting the positions that have been written. In implementation, usually a Pangu file is split into data blocks with variable lengths. For example, chunk 1, chunk 2, chunk 3, and chunk 4. Each chunk will be stored on different ChunkServers in a multi-copy or avoidance manner. For example, chunk 2 will be stored on the corresponding chunk service A, chunk service B, and chunk service C in the form of replicas a, b, and c.

[0216] Optionally, Figure 8 It is a schematic diagram of the Pangu TrimFile capability according to an embodiment of the present application. As Figure 8 shown, the Pangu TrimFile capability can be associated with the Trim operation of a stand-alone storage engine. Generally, a Pangu file is the smallest unit of resource management, and the entire file needs to be deleted for the space to be released. TrimFile, on the other hand, can specify a certain range of data (Range) in the file. For example, for data A, this data A can be deleted without deleting the entire file, that is, the target space can be released. The actual space release operation can be performed by the stand-alone storage engine where the corresponding replica is located, that is, the Trim operation of the ZNS SSD stand-alone storage engine can be called. That is, if it is detected that the space where data A in the Pangu file needs to be released, it can be determined on which chunk data A is located. For example, on chunk 5, and it can be further confirmed in which replicas of the chunk the data is located. For example, data A exists on replicas d, e, and f.

[0217] In this embodiment, the requirement background of the present application is to reduce the cost of quantitatively evaluating the Trim technology. As a new technology provided by Pangu, TrimFile requires a reliable evaluation method to accurately evaluate the functions and technical effects of this technology. The verification and evaluation of this technology are different from the evaluation of the conventional single-function interface of Pangu. That is, it does not rely on evaluating conventional metrics such as data correctness, latency, and IOPS, but depends on the collaboration of the upper business layer of Pangu for evaluation, including: transforming the functional module architecture to access Trim; adjusting and optimizing the garbage collection-related policies of the functional modules; constructing targeted garbage collection scenarios at the business layer for function verification; outputting key metrics related to the functional modules during the test (write amplification, garbage collection traffic); comparing the key metrics under two conditions of using and not using the Trim function in the same test scenario.

[0218] For example, in the EBS scenario, the hardware conditions required for evaluating the Trim technology may include storage servers and stress servers. Five storage servers and one stress server may be required, etc.

[0219] Optionally, Figure 9 It is a schematic diagram of a cluster for evaluating the Trim effect in a related technology according to an embodiment of the present application. As Figure 9 shown, the functional modules required for evaluating the Trim technology may at least include modules such as block services, garbage collection work, Master, data block services, and devices. In this cluster, two main services need to be deployed: block storage services and Pangu services. Relying on at least five functional modules in the two services, they cooperate with each other to evaluate the Trim effect.

[0220] Optionally, Figure 10 It is a schematic diagram of a cluster for evaluating the Trim effect in another related technology according to an embodiment of the present application. As Figure 10 shown, when evaluating the Trim technology effect, two key metrics in the block storage service are mainly concerned, namely write amplification and garbage collection write volume. Two key statistics also need to be concerned, namely the cloud disk write data volume and the garbage collection write data volume. Among them, the cloud disk write data volume (front-end write traffic): is the actual write data volume of cloud disk users, which is statistically counted by the EBSBlockServer module. The garbage collection write data volume: is the data volume read from valid data and written into a new file during the process of recycling garbage data on the cloud disk, which is statistically counted by the EBS GCWorker module.

[0221] Optionally, storage metric: EBS write amplification = (cloud disk write data volume + garbage collection write data volume) / cloud disk write data volume.

[0222] Optionally, Figure 11 It is a schematic diagram of the evaluation result of a TrimFile technology in a full-link test according to an embodiment of the present application. As Figure 11 shown, based on the above method, a comparison of the measured evaluation results can be made. Among them, the evaluation result can be 1M random write, and the result after normalization. Applying the above evaluation method, a comparison of the write amplification and the garbage collection write volume before and after using the Pangu TrimFile technology is made, so as to effectively prove that the application of the Trim technology has a good effect on reducing the write amplification at the service layer and reducing the garbage collection traffic. For example, when the TrimFile is turned off, the write amplification can be 1, and the garbage collection write volume can be 1. However, when the TrimFile is turned on, the write amplification can be 0.64, and the garbage collection write volume can be 0.53.

[0223] However, based on the above method, the evaluation cost is high and the evaluation method is complex. Through the full-link evaluation method, the technical effect of the TrimFile technology at the business level can be accurately evaluated. However, the required hardware conditions and functional module dependencies are relatively heavy. For example, in a physical cluster, 5 storage servers, 40 ZNS SSD disks are required, and two service deployments are also needed, such as the EBS service and the Pangu service. Five key functional modules are also required, such as the EBS BlockServer, EBS GC Worker, Pangu Master, Pangu Worker, and Pangu ChunkServer, etc. An EBS pressure device is also required, such as the VolumeBenchmark. Multiple key parameter adjustments are also needed, such as the parameters related to the trigger and current limiting of EBS garbage collection. Key metrics need to be counted, such as the amount of data written to the cloud disk and the amount of garbage collection data. Therefore, based on the above analysis, it can be seen that the entire evaluation scheme has high hardware costs, many dependent modules, and complex parameter tuning. Therefore, there is still a technical problem of high verification cost for space release operations.

[0224] In the embodiments of the present application, considering the above problems, a Trim technology evaluation method based on a ZNS SSD single-machine storage engine is designed. The new TrimFile technology provided by Pangu has its core implementation as the Trim operation of the single-machine storage engine, which realizes the deletion function for a segment of data. Therefore, the embodiments of the present application shorten the verification link of the technical effect and focus the evaluation on the Trim operation of the single-machine storage engine.

[0225] Optionally, Figure 12 is a schematic diagram of an evaluation system for a Trim technology according to an embodiment of the present application. As Figure 12 shown, the implementation idea of the present application is to analyze the IO load of the EBS functional module, simulate the IO load characteristics of the EBS in a single-machine environment, including the foreground write and garbage collection processes of the cloud disk, and add a process to access and use the Trim operation of the single-machine storage engine to reduce the trigger of garbage collection and reduce the write amplification at the business level, so as to achieve the purpose of verifying the technical effect of the Trim technology, and further achieve the technical effect of reducing the verification cost of the space release operation. As Figure 12 shown, the embodiments of the present application streamline the evaluation dependencies. Using a single-machine pressure device, in a single-machine environment, based on the functional pressure model, the foreground write traffic, garbage collection read and write traffic, and Trim traffic are simultaneously simulated. Based on the above design, the evaluation process for multiple dependencies such as clusters, EBS services, and Pangu services is greatly simplified, and only one ZNS SSD server is needed to run the evaluation test of the TrimFile effect.

[0226] Optionally, Figure 13 Schematic diagram of a functional fuzzy pressure module according to an embodiment of the present application, as Figure 13 shown, the overall module design of the functional simulation pressure module can be composed of a memory index data structure and a storage engine data file. The ChunkTable in the memory data structure may include valid data indexes and garbage data indexes. Among them, the valid data index can maintain the Key of the valid data in each Chunk (corresponding to a data file in the EBS scenario). The garbage data index can maintain the invalid space positions in each Chunk (each segment of invalid space is identified by an offset and a length, that is, Location). By backtracking through the valid data index, the position of the invalid data can also be obtained. The embodiment of the present application adds a garbage data index, which can more quickly find the invalid space in a Chunk, that is, the garbage storage space. The memory data structure may also include a LocationTable, which can be used to maintain the position of the data corresponding to a Key in the Chunk, that is, the offset and data length in the Chunk. The memory data structure may also include a FreeChunkList, which can be used to maintain a list of available Chunks.

[0227] Optionally, as Figure 13 shown, multiple Chunks are maintained in the storage engine data file. The Chunk only supports append writing, and its characteristics are the same as those of the data file in the EBS scenario. The Chunk contains valid data and garbage data associated with the garbage data index included in the garbage space.

[0228] Optionally, the key operation process of the embodiment of the present application may include a writing process. In the writing process, the Chunk may contain multiple data uniquely identified by the Key. The data (Value) is written into the Chunk in append mode through Put(Key,Value), and the ChunkTable and LocationTable in the memory are updated.

[0229] Optionally, the key operation process of the embodiment of the present application may also include an update or delete process. In the update or delete process, if an update or delete operation is performed on the Key, the ChunkTable in the memory can be updated, and the Key can be updated or deleted from the valid data index and LocationTable, and the corresponding data space can be added to the invalid data index, that is, the garbage data index.

[0230] Optionally, Figure 14 Schematic diagram of a garbage collection method for Trim technology according to an embodiment of the present application, as Figure 14 shown, the method may include the following steps:

[0231] Step S1401, delete or update the Key.

[0232] In this embodiment, the Key can be deleted or updated.

[0233] Optionally, when the Key is deleted or updated, it can be stated that there is garbage storage space generated, and step S1402 can be further executed.

[0234] Step S1402, update the garbage ratio.

[0235] In this embodiment, the proportion of garbage space in the current system can be evaluated by counting and updating the garbage ratio, and step S1403 can be further executed.

[0236] Step S1403, determine whether to trigger the Trim process.

[0237] In this embodiment, it can be determined whether to trigger the Trim process. If so, step S1406 can be executed; otherwise, step S1404 can be executed.

[0238] Optionally, by analyzing the current space situation, it can be determined whether to reach the trigger condition for the Trim process. If it is determined that garbage collection is required, for example, if the FreeChunk is insufficient, the garbage collection process needs to be triggered.

[0239] Step S1404, trigger garbage collection.

[0240] In this embodiment, garbage collection can be triggered.

[0241] Optionally, in the garbage collection process, the valid data in a Chunk can be read from the old location and written to the new location in its entirety, and the old Chunk can be deleted to release the storage space occupied by the garbage.

[0242] Step S1405, update the write amplification.

[0243] In this embodiment, the write amplification after the garbage collection operation can be updated.

[0244] Optionally, the write amplification information can also be statistically analyzed periodically and does not necessarily depend on the garbage collection process.

[0245] Step S1406, scan the garbage data index in the chunk table to obtain the garbage space information.

[0246] In this embodiment, if the Trim process is triggered, the garbage data index of the ChunkTable can be scanned to obtain the corresponding garbage space information.

[0247] Optionally, when the trigger Trim condition is reached, the compressor scans the garbage data index in the ChunkTable and constructs a Trim request.

[0248] Step S1407, call the storage engine Trim operation to release the garbage space.

[0249] In this embodiment, the storage engine Trim operation can be called to release the garbage space.

[0250] Optionally, call the Trim interface of the storage engine to send the Trim request to the storage engine. Release the garbage space by calling the stand-alone storage engine.

[0251] Step S1408, the space release operation is successful, and new available chunks are created as needed to support more writes.

[0252] In this embodiment, it can be determined whether the space release operation is successful. If the space is successfully released, new available chunks can be created as needed to support more writes.

[0253] Optionally, confirm whether the space release is successful by calling the interface of the stand-alone storage engine to obtain storage information. After the space release is successful, new writes can be supported according to the remaining space and the remaining available Chunks.

[0254] In the embodiment of the present application, by applying the above evaluation method, the dependencies for verifying the Trim technology are as follows: the number of ZNSSSD servers is reduced by 80%, and only one ZNS SSD server is required; the number of ZNS SSDs is reduced from 60 to 1, from 60 in the whole cluster to at most one ZNS SSD; the memory resource is reduced from 500G+ to 2G, from the total memory resource of 500+ in the whole cluster to 2G of memory resource, and no longer depends on the deployment of EBS and Pangu services.

[0255] Optionally, Figure 15 is a schematic diagram of the verification result of a Trim effect according to an embodiment of the present application, as Figure 15 shown, the technical effect of the embodiment of the present application is that, in a verified scenario that meets the conditions, garbage collection at the service layer can be significantly reduced, and the write amplification can be maintained at an ideal size. As Figure 15 The first line is the verification parameter. In this verification, 80% water level of 300 files with 10M each is simulated, and 10% is reserved for the simulated cloud disk space. The second line is the key statistical result. In the steady state, the write amplification can be maintained at 1.08, which is a scenario that meets the conditions, that is, each garbage space end can be directly released through the Trim technology. In the actual usage scenario, there are more requirements for the continuity of the garbage space, and all actual effects depend on the specific Pattern of the functional module. The garbage ratio (Stales) can be maintained below 5%, and the typical value in the usage scenario is usually 15%. The Trim count (Trims) is the number of Trim requests executed counted within a 5s window. The actual physical space occupied (PhyData), within the storage engine, is the actual physical space occupied.

[0256] Optionally, Figure 16 is a schematic diagram for comparing the write amplification of closing and opening Trim according to an embodiment of the present application. As Figure 16 shown, under the same configuration, for the two cases of opening Trim and closing Trim, the measured write amplification comparisons are as Figure 16 shown, that is, when Trim is opened, the write amplification obtained by using the method of the embodiment of the present application is 0.28, and when Trim is closed, the obtained write amplification is 1.

[0257] In the embodiment of the present application, the method of simulating in a single-machine environment can be used to evaluate the expected effect of the Trim technology. In the functional simulation press, the Trim technology of the ZNS SSD storage engine can be applied, and by comparing the business layer amplification, the function and expected effect of the Trim technology can be verified. Through the above method, the evaluation cost can be significantly reduced by at least more than 80%.

[0258] In an embodiment of the present application, during the process of simulating the load in the storage service scenario where the storage server is located by using a simulation pressure device, the storage space in the storage server can be monitored in real time by the simulation pressure device. When it is detected that there is a garbage storage space in the storage space, garbage data in the to-be-processed data files associated with the load can be found in the garbage storage space, and a garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated according to the garbage data index. In response to the space release request, a corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained and verified to obtain a verification result. The functions and service effects of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present application takes into account that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependencies and shortening the verification link of the effects of this technology is achieved. Furthermore, the technical effect of reducing the verification cost of the space release operation is realized, and the technical problem of high verification cost for the space release operation is solved.

[0259] Embodiment 4

[0260] According to an embodiment of the present application, there is also provided a verification device for the space release operation for implementing the verification method of the space release operation shown above. Figure 3

[0261] Figure 17 FIG. is a schematic diagram of a verification device for the space release operation according to an embodiment of the present application. As Figure 17 shown, the verification device 1700 for the space release operation may include: a first monitoring unit 1702, a first calling unit 1704, a first obtaining unit 1706, and a first verification unit 1708.

[0262] The first monitoring unit 1702 is configured to monitor the space release request of the simulation pressure device. Among them, the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is a garbage storage space in the storage space of the storage server. The garbage data index is used to find garbage data in the to-be-processed data files associated with the load in the garbage storage space.

[0263] The first calling unit 1704 is configured to call an operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0264] ​The first acquisition unit 1706 is configured to acquire storage metrics of a storage space during the process of releasing garbage data in the garbage storage space, where the storage metrics are used to represent the performance of writing data to the storage space.

[0265] The first verification unit 1708 is configured to verify the storage metrics to obtain a verification result.

[0266] Here, the first monitoring unit 1702, the first invocation unit 1704, the first acquisition unit 1706, and the first verification unit 1708 correspond to steps S302 to S308 in Embodiment 1. The instances and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above units may be hardware components or software components stored in a memory (for example, the memory 104) and processed by one or more processors (for example, the processors 102a, 102b..., 102n), and the above units may also be part of a device and can run in the computer terminal 10 provided in Embodiment 7.

[0267] According to an embodiment of the present application, there is also provided a verification device for a space release operation for implementing the above Figure 4 shown space release operation verification method.

[0268] Figure 18 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present application. As Figure 18 shown, the verification device 1800 for the space release operation may include: a second monitoring unit 1802, a second acquisition unit 1804, a first construction unit 1806, and a first sending unit 1808.

[0269] The second monitoring unit 1802 is configured to monitor the storage space of a storage server.

[0270] The second acquisition unit 1804 is configured to acquire a garbage data index corresponding to the garbage storage space when it is detected that there is a garbage storage space in the storage space, where the garbage data index is used to find garbage data in a data file to be processed associated with a load in the garbage storage space.

[0271] The first construction unit 1806 is configured to construct a space release request based on the garbage data index.

[0272] The first sending unit 1808 is configured to send the space release request to an operation interface of a stand-alone storage engine, where the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0273] It should be noted here that the above-mentioned second monitoring unit 1802, second acquisition unit 1804, first construction unit 1806, and first sending unit 1808 correspond to steps S402 to S408 in Embodiment 1. The instances and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned units may be hardware components or software components stored in a memory (for example, memory 104) and processed by one or more processors (for example, processors 102a, 102b..., 102n), and the above-mentioned units may also be part of a device and can run in the computer terminal 10 provided in Embodiment 7.

[0274] According to an embodiment of the present application, there is also provided a verification device for a space release operation for implementing the above Figure 5 verification method of the space release operation shown.

[0275] Figure 19 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present application, as Figure 19 shown. The verification device 1900 for the space release operation may include: a third monitoring unit 1902, a third acquisition unit 1904, a second construction unit 1906, a second call unit 1908, a third acquisition unit 1910, and a second verification unit 1912.

[0276] The third monitoring unit 1902 is configured to monitor the storage space of the storage server.

[0277] The third acquisition unit 1904 is configured to, when it is detected that there is a garbage storage space in the storage space, acquire a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the data file to be processed associated with the load in the garbage storage space.

[0278] The second construction unit 1906 is configured to construct a space release request based on the garbage data index.

[0279] The second call unit 1908 is configured to call an operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0280] The third acquisition unit 1910 is configured to, during the process of releasing the garbage data in the garbage storage space, acquire a storage metric of the storage space, where the storage metric is used to represent the performance of writing data to the storage space.

[0281] The second verification unit 1912 is configured to verify the storage metric to obtain a verification result.

[0282] It should be noted here that the above-mentioned third monitoring unit 1902, third acquisition unit 1904, second construction unit 1906, second invocation unit 1908, third acquisition unit 1910, and second verification unit 1912 correspond to steps S502 to S512 in Embodiment 1. The instances and application scenarios implemented by the six units and the corresponding steps are the same, but are not limited to the content disclosed in the above-mentioned Embodiment 1. It should be noted that the above-mentioned units may be hardware components or software components stored in a memory (for example, memory 2104) and processed by one or more processors (for example, processors 2102a, 2102b..., 2102n). The above-mentioned units may also be part of a device and may run in the computer terminal 210 provided in Embodiment 7.

[0283] In the verification device for the space release operation, during the process of simulating the load in the storage service scenario where the storage server is located by a simulation pressure device, the storage space in the storage server can be monitored in real time by the simulation pressure device. When it is detected that there is a garbage storage space in the storage space, garbage data in the data files to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated according to the garbage data index. In response to the space release request, a corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained and verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present application considers that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependence and shortening the verification link of the effect of this technology is achieved. Furthermore, the technical effect of reducing the verification cost of the space release operation is realized, and the technical problem of high verification cost for the space release operation is solved.

[0284] Embodiment 5

[0285] An embodiment of the present application can provide a computer terminal, and this computer terminal can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the above-mentioned computer terminal can also be replaced with a terminal device such as a mobile terminal.

[0286] Optionally, in this embodiment, the above-mentioned computer terminal can be located in at least one of multiple network devices in a computer network.

[0287] In this embodiment, the computer terminal can execute the program code of the following steps in the verification method for the space release operation: Monitor the space release request of the simulation pressure device. Among them, the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server. The garbage data index is used to find the garbage data in the garbage storage space from the to-be-processed data files associated with the load; Call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; During the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space. Among them, the storage metrics are used to represent the performance of writing data to the storage space; Verify the storage metrics to obtain a verification result.

[0288] Optionally, Figure 20 is a structural block diagram of a computer terminal according to an embodiment of the present application. As Figure 20 shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 2002, a memory 2004, and a transmission device 2006.

[0289] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the verification method and device for the space release operation in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned verification method for the space release operation. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely set relative to the processor, and these remote memories can be connected to the terminal A through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and their combinations.

[0290] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: Call the operation interface to respond to the space release request, and in the garbage storage space, find the initial storage location where the garbage data is located based on the garbage data index; At the initial storage location in the garbage storage space, delete the garbage data.

[0291] Optionally, the above processor can also execute the program code of the following steps: Determine the data blocks including garbage data in the garbage storage space; Identify the valid data in the to-be-processed data files in the data blocks; Read out the valid data from the data blocks; In the case where the read valid data is successfully written to the valid storage space, delete the data blocks including the garbage data at the initial storage location to delete the garbage data.

[0292] Optionally, the above-mentioned processor may further execute program code for the following steps: during the process of releasing garbage data in the garbage storage space, detecting a first amount of written data of valid data written to the valid storage space; and determining a storage metric based on the first amount of written data.

[0293] Optionally, the above-mentioned processor may further execute program code for the following steps: during the process of releasing garbage data in the garbage storage space, detecting a second amount of written data obtained by performing a write operation on the storage space; and determining a storage metric based on the first amount of written data and the second amount of written data.

[0294] Optionally, the above-mentioned processor may further execute program code for the following steps: in response to a shutdown instruction, obtaining an initial storage metric of the storage space in the storage service scenario, where the shutdown instruction is used to indicate prohibiting the invocation of the operation interface; and verifying the storage metric using the initial storage metric to obtain a verification result.

[0295] Optionally, the above-mentioned processor may further execute program code for the following steps: in the case of successfully releasing garbage data in the garbage storage space, updating the valid storage space in the storage space based on the released garbage storage space; and invoking an information interface to perform a write operation on the updated valid storage space.

[0296] The processor may call information and application programs stored in the memory through a transmission device to perform the following steps: monitoring the storage space of the storage server; in the case of detecting a garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the data file associated with the load to be processed in the garbage storage space; constructing a space release request based on the garbage data index; and sending the space release request to the operation interface of the single-machine storage engine; where the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0297] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: monitor the storage space of the storage server; in the case where garbage storage space is detected in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the to-be-processed data file associated with the load in the garbage storage space; construct a space release request based on the garbage data index; call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space; verify the storage metrics to obtain a verification result.

[0298] By adopting the embodiment of the present application, a verification method for space release operation is provided. In the embodiment of the present application, during the process of simulating the load in the storage service scenario where the storage server is located through the simulation pressure device, the storage space in the storage server can be monitored in real time through the simulation pressure device. In the case where garbage storage space is detected in the storage space, the garbage data in the to-be-processed data file associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated according to the garbage data index. In response to the space release request, the corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. During the process of releasing the garbage data, the storage metrics of the storage space can be obtained and verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present application considers that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependence and shortening the verification link of the effect of this technology is achieved. Furthermore, the technical effect of reducing the verification cost of the space release operation is realized, and the technical problem of high verification cost of the space release operation is solved.

[0299] Those of ordinary skill in the art can understand that Figure 20 The structure shown is only for illustration. The computer terminal A can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and a mobile Internet device (abbreviated as MID), a PAD and other terminal devices. Figure 20 It does not limit the structure of the above computer terminal A. For example, the computer terminal A may further include more or fewer components (such as a network interface, a display device, etc.) than those shown in Figure 20 or have the same as Figure 20 The different configurations shown.

[0300] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by a program instructing the hardware related to the terminal device, and the program can be stored in a computer-readable storage medium. The storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0301] Embodiment 6

[0302] The embodiments of the present application further provide a computer-readable storage medium. Optionally, in this embodiment, the above computer-readable storage medium can be used to store the program code executed by the verification method for the space release operation provided in the first embodiment above.

[0303] Optionally, in this embodiment, the above computer-readable storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0304] Optionally, in this embodiment, the computer-readable storage medium is set to store the program code for performing the following steps: monitoring the space release request of the analog pressure device, where the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the analog pressure device detects that there is a garbage storage space in the storage space of the storage server, and the garbage data index is used to find the garbage data in the data file associated with the load to be processed in the garbage storage space; calling the operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtaining the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space; and verifying the storage metrics to obtain a verification result.

[0305] Optionally, the above computer-readable storage medium can also execute the program code for the following steps: calling the operation interface to respond to the space release request, and in the garbage storage space, finding the initial storage location where the garbage data is located based on the garbage data index; and deleting the garbage data at the initial storage location in the garbage storage space.

[0306] Optionally, the above computer-readable storage medium may also execute program code for the following steps: determining data blocks including garbage data in the garbage storage space; identifying valid data in the data blocks in the to-be-processed data file; reading out the valid data from the data blocks; and deleting the data blocks including the garbage data at the initial storage positions to delete the garbage data when the read-out valid data is successfully written into the valid storage space.

[0307] Optionally, the above computer-readable storage medium may also execute program code for the following steps: during the process of releasing garbage data in the garbage storage space, detecting a first amount of written data of the valid data written into the valid storage space; and determining a storage metric based on the first amount of written data.

[0308] Optionally, the above computer-readable storage medium may also execute program code for the following steps: during the process of releasing garbage data in the garbage storage space, detecting a second amount of written data obtained by performing a write operation on the storage space; and determining a storage metric based on the first amount of written data and the second amount of written data.

[0309] Optionally, the above computer-readable storage medium may also execute program code for the following steps: in response to a shutdown instruction for instructing to prohibit calling an operation interface, obtaining an initial storage metric of the storage space in the storage service scenario; and verifying the storage metric by using the initial storage metric to obtain a verification result.

[0310] Optionally, the above computer-readable storage medium may also execute program code for the following steps: when the garbage data in the garbage storage space is successfully released, updating the valid storage space in the storage space based on the released garbage storage space; and calling an information interface to perform a write operation on the updated valid storage space.

[0311] As an optional example, the computer-readable storage medium is configured to store program code for performing the following steps: monitoring the storage space of a storage server; when it is detected that there is a garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the to-be-processed data file associated with a load in the garbage storage space; constructing a space release request based on the garbage data index; and sending the space release request to an operation interface of a stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0312] As an alternative example, a computer-readable storage medium is configured to store program code for performing the following steps: monitoring the storage space of a storage server; in the case where garbage storage space is detected in the storage space, obtaining a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in a to-be-processed data file associated with a load in the garbage storage space; constructing a space release request based on the garbage data index; invoking an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtaining a storage metric of the storage space, where the storage metric is used to represent the performance of writing data to the storage space; and verifying the storage metric to obtain a verification result.

[0313] Embodiment 7

[0314] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0315] Figure 21 It is a block diagram of an electronic device for a method of verifying a space release operation according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0316] As Figure 21 shown, the device 2100 includes a computing unit 2101, which may perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 2102 or a computer program loaded from a storage unit 2108 into a random access memory (RAM) 2103. In the RAM 2103, various programs and data required for the operation of the device 2100 may also be stored. The computing unit 2101, the ROM 2102, and the RAM 2103 are connected to each other through a bus 2104. An input / output (I / O) interface 2105 is also connected to the bus 2104.

[0317] Multiple components in device 2100 are connected to I / O interface 2105, including: an input unit 2106, such as a keyboard, a mouse, etc.; an output unit 2104, such as various types of displays, speakers, etc.; a storage unit 2108, such as a magnetic disk, an optical disc, etc.; and a communication unit 2109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 2109 allows device 2100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0318] The computing unit 2101 can be various general-purpose and / or dedicated processing components with processing and computing capabilities. Some examples of the computing unit 2101 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 2101 executes the various methods and processes described above, such as the verification method for the space release operation. For example, in some embodiments, the verification method for the space release operation can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 2108. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 2100 via the ROM 2102 and / or the communication unit 2109. When the computer program is loaded into the RAM 2103 and executed by the computing unit 2101, one or more steps of the verification method for the space release operation described above can be executed. Alternatively, in other embodiments, the computing unit 2101 can be configured to execute the verification method for the space release operation by any other suitable means (e.g., by means of firmware).

[0319] Embodiment 8

[0320] An embodiment of the present application also provides a chip. Optionally, in this embodiment, the above chip may include a processor, configured to call and run a computer program from a memory, such that a device installed with the chip executes the verification method for the space release operation of the above embodiment of the present application.

[0321] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard parts (ASSP), system-on-a-chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0322] The program code for implementing the methods of this application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0323] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard parts (ASSP), system-on-a-chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0324] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0325] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0326] In order to provide interaction with users, the systems and technologies described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD), a monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with users; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0327] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected with each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0328] A computer system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating blockchain.

[0329] It should be noted that the serial numbers of the embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0330] In the above embodiments of the present application, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0331] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces, and the indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0332] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0333] In addition, in each embodiment of the present application, each functional unit can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0334] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories, random access memories, mobile hard disks, magnetic disks, or optical discs.

[0335] The above are only the preferred embodiments of the present application. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present application.< / location>

Claims

1. A verification method for a space release operation, characterized in that Applied to a single-machine storage engine deployed on a storage server, where a simulation pressure device is also deployed on the storage server, and the simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located. The method includes: Monitoring the space release request of the simulation pressure device, where the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to find the garbage data in the data file to be processed associated with the load in the garbage storage space; Invoking an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; During the process of releasing the garbage data in the garbage storage space, obtaining the storage metrics of the storage space; Verifying the storage metrics to obtain a verification result.

2. The method according to claim 1, wherein Invoking an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space, including: Invoking the operation interface to respond to the space release request, and in the garbage storage space, finding the initial storage location where the garbage data is located based on the garbage data index; Deleting the garbage data at the initial storage location in the garbage storage space.

3. The method according to claim 2, wherein The storage space includes valid storage space other than the garbage storage space. Among them, deleting the garbage data at the initial storage location in the garbage storage space includes: Determining the data block in the garbage storage space that includes the garbage data; Identifying the valid data in the data file to be processed in the data block; Reading out the valid data from the data block; When the read valid data is successfully written into the valid storage space, deleting the data block including the garbage data at the initial storage location to delete the garbage data.

4. The method according to claim 3, wherein During the process of releasing the garbage data in the garbage storage space, obtaining the storage metrics of the storage space, including: During the process of releasing the garbage data in the garbage storage space, detecting the first write data volume of the valid data written into the valid storage space; Determining the storage metrics based on the first write data volume.

5. The method according to claim 4, characterized in that The method further includes: During the process of releasing the garbage data in the garbage storage space, detecting the second write data volume obtained by performing a write operation on the storage space; Determining the storage metrics based on the first write data volume, including: determining the storage metrics based on the first write data volume and the second write data volume.

6. The method according to claim 1, wherein Verifying the storage metrics to obtain a verification result, including: In response to a shutdown instruction, obtaining the initial storage metrics of the storage space in the storage service scenario, where the shutdown instruction is used to indicate prohibiting the invocation of the operation interface; Verify the storage metric using the initial storage metric to obtain the verification result.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: In the case of successfully releasing the garbage data in the garbage storage space, update the available storage space in the storage space based on the released garbage storage space; Call the information interface to perform a write operation on the updated available storage space.

8. A verification method for a spatial release operation, characterized in that, Applied to a simulation pressure device deployed on a storage server, the simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located. A single-machine storage engine is also deployed on the storage server. The method includes: Monitor the storage space of the storage server; In the case of detecting that there is a garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data file to be processed associated with the load in the garbage storage space; Construct a space release request based on the garbage data index; Send the space release request to the operation interface of the single-machine storage engine; Wherein, the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

9. The method according to claim 8, characterized in that, The method further includes: In the case of there being a data identifier for deletion or update in the simulation pressure device, determine that there is a garbage storage space in the storage space, where the data identifier is used to represent the data stored in the storage space.

10. The method according to claim 8, characterized in that, Obtaining the garbage data index corresponding to the garbage storage space includes: Obtain the proportion of the garbage storage space in the storage space; In response to the proportion not meeting the proportion threshold, obtain the garbage data index corresponding to the garbage storage space.

11. The method according to claim 10, wherein The method further includes: In response to the proportion not meeting the proportion threshold, perform a garbage collection operation on the storage space.

12. The method according to claim 8, wherein The storage space includes the available storage space other than the garbage storage space. The method further includes: In the case where the valid data in the data file to be processed is successfully written from the garbage storage space to the available storage space, update the memory index data structure of the simulation pressure device, where the memory index data structure includes the garbage data index and the valid data index of the valid data, and the valid data index is used to find the valid data in the available storage space.

13. The method according to claim 12, wherein The memory index data structure further includes at least one of the following: the position of the garbage data corresponding to the garbage data index in the data block where it is located, the position of the valid data corresponding to the valid data index in the data block where it is located, and the list of valid data blocks where the data block is located.

14. The method according to claim 8, wherein The simulation pressure device is used to store multiple data blocks maintained by the single-machine storage engine.

15. A verification method for a space release operation, characterized in that, Applied to a storage server, the method includes: Monitor the storage space of the storage server; When it is detected that there is a garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the to-be-processed data file associated with the load in the garbage storage space; Construct a space release request based on the garbage data index; Call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; During the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space; Verify the storage metrics to obtain a verification result.

16. A verification system for a space release operation, characterized in that, Comprising: A simulation pressure device for monitoring the storage space; When it is detected that there is a garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the to-be-processed data file associated with the load in the garbage storage space; construct a space release request based on the garbage data index; A single-machine storage engine for calling the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space; verify the storage metrics to obtain a verification result.

17. An electronic device, characterized in that, Comprising: A memory storing an executable program; A processor for running the program, where when the program runs, it executes the method according to any one of claims 1 to 15.

18. A chip, characterized in that, Comprising: A processor configured to call and run a computer program from the memory, so that the device installed with the chip executes the method according to any one of claims 1 to 15.

Images