Verification method and system for operation of freeing up space, electronic device, and chip

Abstract

The present disclosure discloses a verification method and system for an operation of freeing up space, an electronic device, and a chip. The method comprises: monitoring for a space freeing-up request of a simulated pressure device, the space freeing-up request being obtained by construction by the simulated pressure device on the basis of a garbage data index corresponding to garbage storage space when it is detected that the garbage storage space exists in storage space of a storage server, and the garbage data index being used to search the garbage storage space for garbage data in a data file to be processed associated with a load; calling an operation interface of a standalone storage engine to respond to the space freeing-up request, on the basis of the garbage data index, performing an operation of freeing up space on the garbage storage space, and releasing garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, acquiring a storage indicator of the storage space; and verifying the storage indicator to obtain a verification result. According to the present disclosure, the technical problem of high verification costs of the operation of freeing up space is solved.

Description

Space release operation verification method, system, electronic device and chip

[0001] Cross-reference

[0002] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 28, 2023, with application number 2023118500059 and invention name “Verification method, system, electronic device and chip for space release operation”, the entire contents of which are incorporated by reference in this disclosure. Technical Field

[0003] The present disclosure relates to the field of computers, and in particular to a method, system, electronic device, and chip for verifying a space release operation. Background Art

[0004] Currently, TrimFile is a new interface and capability provided by distributed storage systems based on storage servers. It provides fine-grained space resource release capabilities within files for storage server scenarios. For example, TrimFile is an interface and capability provided by Zoned Namespace Solid State Disks (ZNS SSDs). TrimFile can be used to release space from garbage storage space in storage servers. Therefore, it is necessary to test and verify TrimFile technology to ensure the effectiveness of storage services.

[0005] In related technologies, when evaluating TrimFile technology, a full-link evaluation method for storage server scenarios can usually be adopted. This method can usually include steps such as cluster establishment, server deployment, verification scenario construction, and full-link parameter and configuration adjustment. The above method not only has cumbersome evaluation steps and complex coordination, but also relies heavily on hardware and modules for parameter adjustment, resulting in high evaluation costs. Therefore, there is still a technical problem of high verification costs for space release operations.

[0006] To address the above-mentioned problems, no effective solutions have been proposed so far.

[0007] Summary of the Invention

[0008] The embodiments of the present disclosure provide a method, system, electronic device, and chip for verifying a space release operation, so as to at least solve the technical problem of high verification cost for the space release operation.

[0009] According to one aspect of an embodiment of the present disclosure, a method for verifying a space release operation is provided. The method is applied to a stand-alone storage engine deployed on a storage server, and a simulation stressor is also deployed on the storage server, and the simulation stressor is used to simulate the load in the storage service scenario where the storage server is located. The method may include: monitoring the space release request of the simulation stressor, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation stressor detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the data files to be processed that are associated with the load in the garbage storage space; calling the operation interface of the stand-alone storage engine to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verifying the storage index to obtain a verification result.

[0010] According to another aspect of the embodiment of the present disclosure, another verification method for space release operation is provided. The method is applied to a simulation stressor deployed on a storage server, and the simulation stressor is used to simulate the load in the storage service scenario in which the storage server is located, and a stand-alone storage engine is also deployed on the storage server. The method may include: monitoring the storage space of the storage server; in the case of detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; constructing a space release request based on the garbage data index; sending the space release request to the operation interface of the stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0011] According to another aspect of the embodiment of the present disclosure, another verification method for space release operation is provided. The method is applied to a storage server and may include: monitoring the storage space of the storage server; when it is detected that there is garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data file in the garbage storage space; constructing a space release request based on the garbage data index; calling an operation interface to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining a storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verifying the storage index to obtain a verification result.

[0012] According to another aspect of the embodiment of the present disclosure, a verification system for space release operations is also provided. The system may include: a simulator configured to monitor storage space; when detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data files in the garbage storage space; constructing a space release request based on the garbage data index; a stand-alone storage engine configured to call an operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining a storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; and verifying the storage index to obtain a verification result.

[0013] According to another aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device may include a memory and a processor: the memory is configured to store computer-executable instructions, and the processor is configured to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the space release operation verification method of the embodiment of the present disclosure is implemented.

[0014] According to another aspect of an embodiment of the present disclosure, a processor is further provided, wherein the processor is configured to run a program, wherein the method for verifying the space release operation of the embodiment of the present disclosure is executed when the program is running.

[0015] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the space release operation verification method of the present disclosure.

[0016] According to another aspect of the present disclosure, a chip is provided, which includes a processor configured to call and execute a computer program from a memory, so that a device equipped with the chip executes the space release operation verification method of the present disclosure.

[0017] According to another aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, which implements any of the above methods when executed by a processor.

[0018] According to another aspect of an embodiment of the present disclosure, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above methods is implemented.

[0019] According to another aspect of an embodiment of the present disclosure, a computer program is further provided, which implements any of the above methods when executed by a processor.

[0020] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, any of the above methods can be implemented.

[0021] In the embodiment of the present disclosure, in the process of simulating the load in the storage service scenario in which the storage server is located by using a simulated stressor, the storage space in the storage server can be monitored in real time by using the simulated stressor. When it is detected that there is garbage storage space in the storage space, the garbage data in the data file to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. The corresponding space release request can be generated based on the garbage data index. In response to the space release request, the operation interface of the stand-alone storage engine is called, and the space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data. In the process of releasing garbage data, the storage index of the storage space can be obtained, and the performance of writing data to the storage space can be evaluated by the storage index, and the storage index can be verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated by the verification result. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFlie technology can be simplified by the above steps, the purpose of streamlining the evaluation dependency and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0022] It is easy to note that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present disclosure, and do not constitute a limitation of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0024] FIG1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a verification method for a space release operation according to an embodiment of the present disclosure;

[0025] FIG2 is a structural block diagram of a computing environment of a method for verifying a space release operation according to an embodiment of the present disclosure;

[0026] FIG3 is a flowchart of a method for verifying a space release operation according to an embodiment of the present disclosure;

[0027] FIG4 is a flowchart of another method for verifying a space release operation according to an embodiment of the present disclosure;

[0028] FIG5 is a flowchart of another method for verifying a space release operation according to an embodiment of the present disclosure;

[0029] FIG6 is a flowchart of a system for verifying a space release operation according to an embodiment of the present disclosure;

[0030] FIG7 is a schematic diagram of a deployment of Pangu files according to an embodiment of the present disclosure;

[0031] FIG8 is a schematic diagram of Pangu TrimFile capability according to an embodiment of the present disclosure;

[0032] FIG9 is a schematic diagram of a cluster for evaluating Trim effect in a related art according to an embodiment of the present disclosure;

[0033] FIG10 is a schematic diagram of a cluster for evaluating the Trim effect in another related technology according to an embodiment of the present disclosure;

[0034] FIG11 is a schematic diagram of an evaluation result of a TrimFile technology in a full-link test according to an embodiment of the present disclosure;

[0035] FIG12 is a schematic diagram of an evaluation system of the Trim technology according to an embodiment of the present disclosure;

[0036] FIG13 is a schematic diagram of a functional fuzzy stressor module according to an embodiment of the present disclosure;

[0037] FIG14 is a schematic diagram of a garbage collection method using the Trim technology according to an embodiment of the present disclosure;

[0038] FIG15 is a schematic diagram of a verification result of a Trim effect according to an embodiment of the present disclosure;

[0039] FIG16 is a schematic diagram showing a comparison of Trim write amplification with and without Trim according to an embodiment of the present disclosure;

[0040] FIG17 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present disclosure;

[0041] FIG18 is a schematic diagram of another verification device for space release operation according to an embodiment of the present disclosure;

[0042] FIG19 is a schematic diagram of another verification device for space release operation according to an embodiment of the present disclosure;

[0043] FIG20 is a structural block diagram of a computer terminal according to an embodiment of the present disclosure;

[0044] FIG21 is a block diagram of an electronic device according to a method for verifying a space release operation according to an embodiment of the present disclosure;

[0045] FIG22 is a structural block diagram of a processor according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0047] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential sequence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or apparatus.

[0048] First, some nouns or terms that appear in the description of the embodiments of the present disclosure are subject to the following explanations:

[0049] A distributed storage system stores data on multiple independent storage servers and manages services and data over a network. Multiple copies provide high reliability, high availability, and high scalability. Common distributed storage systems include the highly fault-tolerant Hadoop Distributed File System (HDFS), Amazon Simple Storage Service (Amazon 3S), and Ceph.

[0050] Storage cluster: A distributed storage system links multiple storage servers into a storage cluster. A typical storage cluster can contain thousands of storage servers.

[0051] A standalone storage engine is a service process running on each storage server, providing storage services for that server. It can include high-performance read and write, user input / output (IO) request scheduling, data correctness scanning, and abnormal data reporting. In the Pangu distributed storage system, a standalone storage engine refers to a ChunkServer. In this embodiment, a standalone storage engine is a customized standalone storage engine based on the ZNS SSD.

[0052] Pangu files provide storage engine write-link (append-only) file semantics to functional modules. This means that only appending is supported, and overwriting previously written locations is not supported. In practice, a Pangu file is typically split into data blocks of variable length, also known as chunks. Each chunk is stored in multiple copies or encoded on different chunk servers.

[0053] The TrimFile capability is similar to the Trim operation of the stand-alone storage engine. Typically, a Pangu file is a small unit of resource management. The entire file must be deleted to free up space. The TrimFile capability allows you to delete a specific data range in a file without deleting the entire file, thus freeing up garbage storage space. The actual space release operation is performed by the stand-alone storage engine where the corresponding replica resides, namely, calling the Trim operation of the ZNS SSD stand-alone storage engine.

[0054] Elastic Block Storage (EBS) service provides users with cloud disks;

[0055] Cloud disk garbage data. Since EBS is based on the Log-Structured Merge Tree (LSM Tree) architecture, new data is written to the Pangu file in an append-only manner. When using the cloud disk, it is only part of the data in the Pangu file.

[0056] EBS garbage collection: EBS cloud disk data corresponds to multiple Pangu files. Valid data is read from the Pangu files, written to new Pangu files, and then the old Pangu files are deleted. This deletes garbage data and frees up storage space. This is called EBS garbage collection.

[0057] According to an embodiment of the present disclosure, a method for verifying a space release operation is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0058] The method embodiment provided in Example 1 of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a verification method for a space release operation according to an embodiment of the present disclosure. As shown in Figure 1, the computer terminal 10 (or mobile device) may include one or more (102a, 102b, ..., 102n are used in the figure to illustrate) processors 102 (the processor 102 may include but is not limited to a microprocessor (Microcontroller Unit, referred to as MCU) or a programmable logic device (Field Programmable Gate Array, referred to as FPGA) and other processing devices), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that the structure shown in Figure 1 is only for illustration and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1 , or have a configuration different from that shown in FIG. 1 .

[0059] The hardware structure block diagram shown in Figure 1 can not only serve as an exemplary block diagram of the above-mentioned computer terminal 10 (or mobile device), but also as an exemplary block diagram of the above-mentioned server. In an optional embodiment, Figure 2 shows in a block diagram an embodiment of using the computer terminal 10 (or mobile device) shown in Figure 1 as a computing node in the computing environment 201.

[0060] FIG2 is a block diagram of a computing environment for a method for verifying a space release operation according to an embodiment of the present disclosure. As shown in FIG2 , a computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ... in the figure). The computing nodes all contain local processing and memory resources, and the end user 202 can remotely run applications or store data in the computing environment 201. The application can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in the computing environment 201, representing services "A", "D", "E" and "H" respectively.

[0061] End user 202 can provide and access services through a web browser or other software application on a client. In some embodiments, the provisioning and / or request of end user 202 can be provided to the ingress gateway 230. The ingress gateway 230 may include a corresponding agent to handle the provisioning and / or request for services (one or more services provided in the computing environment 201).

[0062] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can simulate a real computer by initializing a virtual machine, executing programs and applications without directly contacting any actual hardware resources. While virtual machines virtualize machines, according to container-based virtualization, containers can be started to virtualize the entire operating system so that multiple workloads can run on a single operating system instance.

[0063] In one embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). For example, as shown in Figure 2, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). The Pod may include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in the Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with Pods similar to Pods.

[0064] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201. Executing one or more functions of one service may require invoking one or more functions of another service. As shown in FIG2 , service "A" 220-1 receives a user request from end user 202 from ingress gateway 230. Service "A" 220-1 may invoke service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to execute one or more functions.

[0065] This computing environment can be a cloud computing environment, where resource allocation is managed by the cloud service provider, allowing for feature development without having to worry about implementing, adjusting, or scaling servers. This computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be partitioned to perform a set of functions that can scale independently and automatically, rather than scaling a single hardware device to handle the potential load.

[0066] In the above operating environment, the present disclosure provides a method for verifying a space release operation as shown in FIG3 . This method can be deployed in a stand-alone storage engine on a storage server. It should be noted that the method for verifying a space release operation in this embodiment can be executed by the mobile terminal in the embodiment shown in FIG1 . FIG3 is a flow chart of a method for verifying a space release operation according to an embodiment of the present disclosure. As shown in FIG3 , the method may include the following steps:

[0067] Step S302, monitor the space release request of the simulated stressor, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulated stressor detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space.

[0068] In the technical solution provided in the above step S302 of the present disclosure, the space release request of the simulated stressor can be monitored, wherein the storage server can be called a cluster, which can be a distributed storage system, for example, a Pangu distributed file system, which can include a ZNS SSD. The stand-alone storage engine can be a stand-alone storage engine of Pangu, for example, Pangu data block service (PanguChunkServer). A simulated stressor can also be deployed on the storage server. The simulated stressor can be used to at least simulate the load in the storage service scenario in which the storage server is located, and can be a functional simulated stressor, for example, a stand-alone stressor, in which a functional stress model can be deployed. The storage service scenario can be an upper-layer function, a usage scenario of the LSM Tree architecture, or an EBS functional module. The load can be called a load characteristic, such as an IO load characteristic. The simulated stressor can include a target data index and a garbage data index. The garbage data index can be represented as an identifier (key) of the garbage data contained in the chunk table. The chunk table can also include a target data index. The storage space is the storage space of the storage device in the storage server. The storage engine data can include target data and garbage data. When the stress simulator detects the presence of garbage storage space in the storage server's storage space, it generates a space release request based on the garbage data index corresponding to the garbage storage space. The space release request can be a Trim request. The garbage storage space can be Trim garbage space, also known as invalid space. The garbage data index, also known as the invalid data index, can be used to locate garbage data in the pending data files associated with the workload in the garbage storage space. The pending data files can be Pangu files.

[0069] Optionally, a stress simulator on the storage server can be used to simulate at least the load characteristics of the storage service scenario. Furthermore, during the simulation, the stress simulator can also monitor the storage space of the memory server in real time. When junk data is detected in the load-related pending data files in the storage space, a corresponding space release request can be constructed based on the corresponding junk data index.

[0070] Optionally, based on the IO load analysis of the EBS functional modules, a standalone stressor can be used to simulate the IO load characteristics of EBS in a standalone environment. This simulation can include simulating the cloud disk frontend write and garbage collection flows. Additional processes can be added to simulate processes such as the Trim operation of the standalone storage engine. It should be noted that the processes simulated by the stressor are for illustrative purposes only and are not specifically limited here.

[0071] In the related art, the process of verifying TrimFile through steps such as cluster building, server deployment, verification scenario construction, and full-link parameter and configuration adjustment is not only cumbersome and complex to coordinate, but also usually includes building 10 clusters for verification. Therefore, there is still a technical problem of low efficiency and high cost in verifying the space release operation. However, in the embodiment disclosed herein, the evaluation dependency of TrimFile is simplified, and a stand-alone stressor is used to simulate the load conditions of the storage service scenario in a stand-alone environment (single storage node). During the simulation, the process of the space release operation can be verified, thereby greatly simplifying the evaluation process's dependencies on multiple clusters, EBS services, and Pangu services. Therefore, only one ZNS SSD server is needed to run the evaluation test process of the space release operation triggered by TrimFile, which not only simplifies the evaluation process but also reduces the number of servers used, thereby achieving the technical effect of improving the verification efficiency of the space release operation and reducing costs.

[0072] Step S304: calling the operation interface of the stand-alone storage engine to respond to the space release request, performing a space release operation on the garbage storage space based on the garbage data index, and releasing the garbage data in the garbage storage space.

[0073] In the technical solution provided in the above step S304 of the present disclosure, after monitoring the space release request of the simulated pressure device, the operation interface of the stand-alone storage engine can be called to respond to the space release request, and the space release operation can be performed on the garbage storage space based on the garbage data index to release the corresponding garbage data in the garbage storage space, wherein the operation interface can be a new interface, such as the Trime operation interface.

[0074] Optionally, after detecting that the simulation pressure device has a space release request, the operation interface of the stand-alone storage engine can be called to filter out garbage data from the data file to be processed by the garbage data index corresponding to the garbage data to be released contained in the space release request, and the garbage data can be released from the garbage storage space where it is located.

[0075] For example, this embodiment targets the TrimFile technology, and its core implementation is the Trim operation of a stand-alone storage engine, which realizes the deletion function for a section of data. That is, after detecting a space release request, the operation interface of the stand-alone storage engine can be called to determine the corresponding garbage data for deletion based on the garbage data index in the space release request, thereby releasing the garbage storage space for storing the garbage data.

[0076] Step S306: in the process of releasing the garbage data in the garbage storage space, obtaining the storage index of the storage space.

[0077] In the technical solution provided in the above step S306 of the present disclosure, when calling the operation interface of the stand-alone storage engine to respond to the space release request, a release operation is performed on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space. In the process of releasing the garbage data in the garbage storage space, the storage index in the storage space can be obtained, wherein the storage index can be used to represent the performance of writing data to the storage space, and the garbage collection traffic after the TrimFile function in the embodiment of the present disclosure is started is evaluated by determining the performance of writing data. For example, the storage index can be write amplification information, and the data written in the storage space can be summarized, calculated and analyzed by the write amplification information to amplify the data, so as to better display the trends and characteristics of the data, to determine the performance of writing data to the storage space, and thus to evaluate whether the garbage collection traffic is reduced after starting TrimFile. In the embodiment of the present disclosure, the write amplification information can also be referred to as (functional level) write amplification, such as write amplification in the EBS function scenario (EBS write amplification).

[0078] It should be noted that the above-mentioned storage indicators are write amplification information for illustration only and are not specifically limited here. As long as the storage indicators can determine the performance of writing data in the storage space and thus evaluate whether the garbage collection flow is reduced, they are within the protection scope of the embodiments of the present disclosure.

[0079] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called. The data obtained in the current release process can be compared with the business layer write amplification to verify the function and expected application effect of the Trime technology.

[0080] Step S308: Verify the storage indicator to obtain a verification result.

[0081] In the technical solution provided in the above step S308 of the present disclosure, in the process of releasing garbage data in the garbage storage space, after obtaining the storage index of the storage space, the storage index can be verified to obtain a verification result, wherein the verification result can be used to represent the effect of the space release operation, that is, it can be used to represent the function and expected application effect of the Trim technology.

[0082] Optionally, the storage index during the process of releasing the garbage data is verified to determine whether the function of the Trim technology can achieve the expected application effect.

[0083] For example, under the same configuration, you can collect business layer write amplification when Trim is enabled to release garbage data, or you can pre-collect business layer write amplification when Trim is disabled. By comparing the two business layer write amplifications, you can determine whether the Trim technology can achieve the expected results.

[0084] It should be noted that the above process and method of analyzing whether the Trim technology meets the expected effect are only examples and are not specifically limited here.

[0085] In the disclosed embodiment, the simulated stressor in the storage server is used to simulate the load in the storage service scenario in a stand-alone environment, and the Trim technology of the ZNS SSD storage engine can be applied in the simulated stressor. By comparing the business layer write method, the function and expected application effect of the Trim technology can be verified, thereby improving the verification efficiency of the space release operation. In addition, in this process, the evaluation dependency is streamlined, and the simulated stressor is used in a stand-alone environment based on the functional pressure model to reduce the triggering of garbage collection and reduce the business layer write amplification, thereby achieving the technical effect of reducing the verification cost of the space release operation.

[0086] Through the above steps S302 to S308 of the present disclosure, in the process of simulating the load in the storage service scenario where the storage server is located by using a simulated stressor, the storage space in the storage server can be monitored in real time by using a simulated stressor. When it is detected that there is garbage storage space in the storage space, the garbage data in the to-be-processed data file associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. The corresponding space release request can be generated according to the garbage data index. In response to the space release request, the operation interface of the stand-alone storage engine is called, and the space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data. In the process of releasing garbage data, the storage index of the storage space can be obtained, the performance of the storage space writing data can be evaluated by the storage index, and the storage index can be verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFlie technology can be simplified through the above steps, the purpose of streamlining the evaluation dependencies and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0087] The above method of this embodiment is further introduced below.

[0088] As an optional implementation, step S304 calls the operation interface of the stand-alone storage engine to respond to the space release request, performs a space release operation on the garbage storage space based on the garbage data index, and releases the garbage data in the garbage storage space, including: calling the operation interface to respond to the space release request, searching for the initial storage location of the garbage data in the garbage storage space based on the garbage data index; and deleting the garbage data at the initial storage location in the garbage storage space.

[0089] In this embodiment, when calling the operation interface corresponding to the space release request, based on the garbage data index, the space release operation is performed on the garbage storage space. In the process of releasing the garbage data in the garbage storage space, the operation interface of the stand-alone storage engine can be called to respond to the space release request. In the garbage storage space, the initial storage position of the garbage data is found based on the garbage data index, and the garbage data is deleted at the initial storage position in the garbage storage space, where the initial position can also be called the old position or the designated position.

[0090] Optionally, after monitoring a space release request, an operation interface can be called to analyze the space release request, and the garbage data index corresponding to the garbage data on which the space release operation is to be performed can be analyzed from the space release request. The initial storage location of the garbage data can be found through the garbage data index, and the garbage data can be deleted at the initial storage location.

[0091] In related technologies, the target data in a Chunk can be read from the initial position, all written to the new position, and the Chunk at the initial position can be deleted by triggering the garbage collection process to release the storage space occupied by the garbage. However, the embodiment of the present disclosure is different from the above-mentioned garbage collection. The Trim technology can be used to delete the garbage data at a specified position in the Chunk to release the garbage storage space there, while avoiding data recovery reading and writing, thereby reducing the garbage collection flow and reducing write amplification, thereby solving the technical problem of high verification cost of the space release operation.

[0092] As an optional implementation, the storage space includes a target storage space, which is the remaining storage space in the storage space except the garbage storage space, wherein the garbage data is deleted at the initial storage position in the garbage storage space, including: determining the data block including the garbage data in the garbage storage space; identifying the target data in the data file to be processed in the data block; reading the target data from the data block; and when the read target data is successfully written to the target storage space, deleting the data block including the garbage data at the initial storage position to delete the garbage data.

[0093] In this embodiment, the storage space may also include a target storage space, which is the remaining storage space in the storage space excluding the garbage storage space. At the initial position in the garbage storage space, during the process of deleting garbage data, the data block where the garbage data is located in the garbage storage space can be determined. The target data in the data file to be processed can be identified in the data block, the target data can be read from the data block, and the valid processing read out can be written to the corresponding target storage position in the target storage space, and after the target data is successfully written to the target storage position, the data block including the garbage data at the initial storage position can be deleted to delete the garbage data, wherein the target data can be the remaining data excluding the garbage data in the data file to be processed, and the data block including the garbage data at the initial storage position can be an old chunk. The target storage space can be available space.

[0094] Optionally, through the functional simulator, in a single storage node environment, based on the functional stress model, the foreground write traffic, garbage collection read and write traffic, and Trim traffic are simulated simultaneously.

[0095] Optionally, a storage engine data file can include multiple chunks. Chunks can only support append-only writes, and their characteristics are consistent with data files in EBS scenarios. Chunks can contain target data and garbage data in the garbage storage space.

[0096] Optionally, a Chunk can contain multiple data with unique identification keys. By marking (Put) the key of the data (Value), such as junk data and target data, the append mode is written to the Chunk, and the ChunkTable in the storage space and the Location Table in the local area can be updated.

[0097] Optionally, during the update or deletion process, if the Key is updated or deleted, it is necessary to update or delete the ChunkTable in the storage space, and the Key from the target data index and LacationTable, and add the corresponding data space to the invalid data index.

[0098] Optionally, after detecting a space release request, it can determine whether to execute the Trim process. If so, it can scan the ChunkTable garbage data index to obtain garbage space information. The standalone storage engine Trim operation can be called to release garbage storage space. After the garbage storage space is successfully released, new available Chunks can be created on demand to support more writes.

[0099] As an optional implementation, in the process of releasing garbage data in the garbage storage space, the storage index of the storage space is obtained, including: in the process of releasing garbage data in the garbage storage space, detecting the first write data amount of the target data written to the target storage space; and determining the storage index based on the first write data amount.

[0100] In this embodiment, during the process of releasing garbage data from the garbage storage space, a first written data volume of the target data written to the target storage space may be detected, and a storage indicator may be determined based on the first written data volume, wherein the first written data volume may be the amount of data written for garbage collection. The target storage location may be used to indicate a new file to which the target data is written.

[0101] Optionally, when a Key is deleted or updated, it indicates that garbage storage space is generated. The proportion of garbage storage space in the current system can be evaluated by counting and updating the garbage ratio. By analyzing the current space situation, it can be determined whether the Trim process has been triggered. If it is determined that garbage collection is required, the garbage collection process needs to be triggered. For example, garbage collection is required when there are insufficient free data blocks (Free Chunk) in the free block list (FreeChunkList). The first written data amount can be determined to update the storage index, that is, the write amplification information can be updated. The write amplification information can be statistically analyzed periodically and does not necessarily rely on the energy recovery process.

[0102] Optionally, when the Trim triggering condition is reached, the simulator can scan the garbage index data in the ChunkTable to construct a Trim request.

[0103] Optionally, the Trim operation interface of the storage engine can be called to send a Trim request to the storage engine.

[0104] Optionally, in the process of releasing the garbage data corresponding to the space release request, the interface for obtaining storage information of the single-machine storage epidemic is called to confirm whether the garbage storage space is released successfully. After the space is released successfully, new writing can be supported according to the remaining space and the remaining available chunks.

[0105] Optionally, in the process of releasing garbage data, the target data may be read and written to the data amount in the new file, thereby obtaining the first written data amount.

[0106] For example, the amount of first written data may be counted by a garbage collection worker (GC Worker) in the EBS.

[0107] As an optional implementation, the method also includes: in the process of releasing garbage data in the garbage storage space, detecting a second amount of written data obtained by performing a write operation on the storage space; determining a storage index based on the first amount of written data, including: determining the storage index based on the first amount of written data and the second amount of written data.

[0108] In this embodiment, in the process of releasing garbage data in the garbage storage space, the second write data volume obtained by performing a write operation in the storage space can also be detected, and the storage index can be determined by the first write data volume and the second write data volume, wherein the second write data volume can be the cloud disk write data volume.

[0109] Optionally, during the process of releasing garbage data, the actual amount of data written by the user in the cloud disk may be detected to obtain a second amount of written data.

[0110] For example, the second write data volume can be counted through the BlockServer module in EBS.

[0111] Optionally, after the first amount of written data and the second amount of written data are determined, a storage index may be calculated based on the first amount of written data and the second amount of written data.

[0112] For example, after obtaining the second write data volume in the above manner, the ESB write amplification can be determined according to the following formula, that is, EBS write amplification = (cloud disk write data volume + garbage collection write data volume) / cloud disk write data volume.

[0113] As an optional implementation, step S308 verifies the storage indicators and obtains verification results, including: in response to a shutdown instruction, obtaining the initial storage indicators of the storage space in the storage service scenario, wherein the shutdown instruction can be used to indicate that calling the operation interface is prohibited; and using the initial storage indicators to verify the storage indicators to obtain verification results.

[0114] In this embodiment, in the process of verifying the storage index and obtaining the verification result, the operation interface can be prohibited from being called based on the shutdown instruction. In this case, the initial storage index of the storage space in the storage service scenario can be obtained, and then the storage index can be verified using the initial storage index to obtain the verification result, wherein the shutdown instruction can be used to indicate the prohibition of calling the operation interface. The operation interface can be the operation interface corresponding to the TrimFile function. The initial storage index can be used to represent the write amplification when the TrimFile function is turned off.

[0115] Optionally, when TrimFile is turned off, the initial storage index obtained by releasing garbage data in the garbage storage space during the garbage collection process can be collected, where the initial storage index can be the write amplification detected when the TrimFile function of the distributed storage system is turned off.

[0116] Optionally, when TrimFile is turned on, the storage index obtained by releasing garbage data in the garbage storage space during the garbage collection process can be collected, wherein the above storage index can be the write amplification detected when the TrimFile function of the distributed storage system is turned on.

[0117] Optionally, the above two storage indicators can be used to determine whether the TrimFile function can achieve the expected effect.

[0118] For example, if the write amplification detected when the TrimFile function of the distributed storage system is turned on is greater than the write amplification detected when the TrimFile function of the distributed storage system is turned off, it can be said that starting the TrimFile function has a certain effect on reducing the amount of garbage collection writes.

[0119] For example, you can compare the write amplification with and without the TrimFile function to determine whether the write amplification with TrimFile enabled is smaller than when TrimFile is disabled. If so, the TrimFile function has achieved the expected effect. Otherwise, the TrimFile function has not achieved the expected effect. It should be noted that this is only an example and does not impose specific restrictions on the process and method for determining whether the TrimFile function has achieved the expected effect.

[0120] As an optional implementation, the method further includes: when the garbage data in the garbage storage space is successfully released, updating the target storage space in the storage space based on the released garbage storage space; and calling the information interface to perform a write operation on the updated target storage space.

[0121] In this embodiment, when the garbage data in the garbage storage space is successfully released, the target storage space in the storage space can be updated based on the released garbage storage space, and the information interface can be called to perform a write operation to the updated target storage space to write data, wherein the information interface can be an interface for obtaining storage information in a stand-alone storage engine.

[0122] Optionally, whether the garbage data is successfully released can be detected. When it is detected that the garbage data is successfully released based on the space release request, the garbage storage space after the garbage data is released can be updated, that is, the garbage storage space where the garbage data has been released can be updated to the target storage space to write the data that needs to be written subsequently.

[0123] Alternatively, the storage information acquisition interface in the standalone storage engine can be used to detect whether the garbage data has been successfully released. After confirming that the garbage data has been successfully released, the garbage storage space can be adjusted to the target storage space. New writes can be performed based on the remaining space and remaining available chunks.

[0124] The embodiment of the present disclosure further provides a method for verifying a space release operation from the simulated pressure side. FIG4 is a flow chart of a method for verifying a space release operation according to an embodiment of the present disclosure. As shown in FIG4 , the method is applied to a simulated pressure device deployed on a storage server. The simulated pressure device is used to simulate the load in a storage service scenario in which the storage server is located. A stand-alone storage engine is also deployed on the storage server. The method may include the following steps:

[0125] Step S402: monitor the storage space of the storage server.

[0126] In the technical solution provided in the above step S402 of the present disclosure, the storage space in the storage server can be monitored by simulating a pressure device.

[0127] Optionally, a simulated stressor on the storage server is used to simulate at least the load characteristics in the storage service scenario. During the simulation process, the simulated stressor can also be used to perform real-time detection on the storage space of the memory server.

[0128] Optionally, based on EBS IO load analysis, a single-machine stressor can be used to simulate EBS IO load characteristics in a single-machine environment. This simulation can include simulating the cloud disk frontend write and garbage collection flows. You can also add processes to simulate processes such as trim operations using a single-machine storage engine.

[0129] In the disclosed embodiment, the evaluation dependency of TrimFile is simplified, and a stand-alone stressor is used to simulate the load conditions of the storage service scenario in a stand-alone environment. During the simulation, the process of the space release operation can be verified, thereby greatly simplifying the evaluation process's dependencies on clusters, EBS services, and Pangu services. As a result, only one ZNS SSD server is needed to run the evaluation test process of the space release operation triggered by TrimFile, which not only simplifies the evaluation process but also reduces the number of servers used, thereby achieving the technical effect of improving the verification efficiency of the space release operation and reducing costs.

[0130] Step S404 , when it is detected that there is a garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space is obtained, wherein the garbage data index is used to search for garbage data in the to-be-processed data file associated with the load in the garbage storage space.

[0131] In the technical solution provided in the above step S404 of the present disclosure, when it is detected that there is garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space can be obtained, wherein the garbage data index can be used to search for garbage data in the load-associated data files to be processed in the garbage storage space.

[0132] Optionally, when garbage data is found in the load-related to-be-processed data files in the monitoring storage space, a corresponding space release request may be constructed according to the corresponding garbage data index.

[0133] When garbage data is found in the load-related to-be-processed data files in the monitoring storage space, a corresponding space release request can be constructed according to the corresponding garbage data index.

[0134] Step S406: construct a space release request based on the garbage data index.

[0135] In the technical solution provided in the above step S406 of the present disclosure, after detecting that there is garbage storage space in the storage space and obtaining the garbage data index corresponding to the garbage storage space, a space release request can be constructed based on the garbage data index.

[0136] Optionally, a space release request is constructed based on a garbage data index, that is, the space release request may store a garbage data index corresponding to the garbage data to be released.

[0137] Optionally, it is monitored whether there is garbage storage space in the storage space. When it is detected that there is garbage storage space, the garbage data index corresponding to the garbage storage space can be obtained from the chunk table. The garbage index data may include the chunk identifier (Chunkid) in the chunk table and the location of the corresponding garbage data (List <location>). After obtaining the above Chunkid and List <location>Afterwards, it can be filled in the space release request, which can be sent to the stand-alone storage engine to prompt the stand-alone storage engine where the garbage data to be pruned is located.

[0138] It should be noted that the garbage data index used to determine the garbage data to be modified in the above-mentioned construction of the space release request is only for example and is not specifically limited here. As long as the space release request can reflect the location of the garbage data and instruct the stand-alone storage engine to construct it, it is within the protection scope of the embodiment of the present disclosure.

[0139] Step S408, sending a space release request to the operation interface of the stand-alone storage engine, wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0140] In the technical solution provided in the above step S408 of the present disclosure, after a space release request is constructed based on the garbage data index, the space release request can be sent to the operation interface of the stand-alone storage engine, wherein the space release request can be responded to by the operation interface, and the garbage data index in the space release request can be used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the corresponding garbage data in the garbage storage space, and in the process of releasing the garbage data in the garbage storage space, the storage indicators of the storage space are verified to obtain a verification result.

[0141] Optionally, a space release request can be sent to an operation interface through a simulation pressure device. The corresponding garbage data can be released through the operation interface, and the release process can be verified to obtain a verification result.

[0142] Optionally, after detecting that the simulation pressure device has a space release request, the operation interface can be called to filter out garbage data from the data file to be processed by the garbage data index corresponding to the garbage data to be released contained in the space release request, and the garbage data can be released from the garbage storage space where it is located.

[0143] For example, for TrimFile technology, the core implementation is the Trim operation of a stand-alone storage engine, which realizes the deletion function for a section of data. That is, after detecting a space release request, the operation interface can be called to determine the corresponding garbage data for deletion based on the garbage data index in the space release request, thereby releasing the garbage storage space for storing the garbage data.

[0144] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called. The data obtained in the current release process can be compared with the business layer write amplification to verify the function and expected application effect of the Trime technology.

[0145] Optionally, the storage indicators during the process of releasing the garbage data are verified to determine whether the function of the Trim technology can achieve the expected application effect.

[0146] For example, under the same configuration, you can collect business layer write amplification when Trim is enabled to release garbage data, or you can pre-collect business layer write amplification when Trim is disabled. By comparing the two business layer write amplifications, you can determine whether the Trim technology can achieve the expected results.

[0147] Through the above steps S402 to S408 of the present disclosure, the storage space of the storage server is monitored; when it is detected that there is garbage storage space in the storage space, the garbage data index corresponding to the garbage storage space is obtained, wherein the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; a space release request is constructed based on the garbage data index; the space release request is sent to the operation interface of the stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0148] The above method of this embodiment is further introduced below.

[0149] As an optional implementation, the method may include: determining that there is garbage storage space in the storage space when there is a deleted or updated data identifier in the simulation pressure device, wherein the data identifier is used to represent the data stored in the storage space.

[0150] In this embodiment, when there is a deleted or updated data identifier in the simulated pressure device, the garbage storage space in the storage space can be determined, wherein the data identifier can be used to identify the data stored in the storage space. The data identifier can be a key, which can be a unique identifier of the data stored in the storage space.

[0151] Optionally, in the process of monitoring the storage space in the storage server through a simulated stressor, the simulated stressor can be used to monitor whether there are deleted or updated data identifiers in the storage server. When a deleted or updated data identifier is detected, it can be determined from the storage space whether there is garbage storage space.

[0152] Optionally, when the simulation pressure device detects that a key is deleted or updated in the storage server, it can be indicated that garbage space is generated, and it is necessary to release the garbage data in the garbage space by constructing a corresponding space release request.

[0153] As an optional implementation, step S404, obtaining a garbage data index corresponding to the garbage storage space, includes: obtaining a proportion of the garbage storage space to the storage space; and in response to the proportion meeting a proportion threshold, obtaining a garbage data index corresponding to the garbage storage space.

[0154] In this embodiment, in the process of obtaining the garbage data index corresponding to the garbage storage space, it can be determined whether the garbage storage space meets the target condition. The target condition can be whether the proportion of the garbage storage space to the storage space meets the proportion threshold. If not, the garbage data index corresponding to the garbage storage space can be obtained, wherein the target condition can be used to indicate whether the garbage storage space has reached the triggering Trim process. The proportion can also be called the garbage ratio (Stales). The proportion threshold can be a value set in advance according to the size of the storage space of the storage device in the storage server, or it can be a value set by itself according to the actual demand for space release. For example, the value can be pre-set to 5% or 15%. It should be noted that the size and setting method of the above-mentioned proportion threshold are only for example purposes and are not specifically limited here.

[0155] Optionally, by analyzing the current storage space situation in the storage server, it is determined whether the Trim process is triggered.

[0156] For example, if the target condition is to determine whether there are enough FreeChunks in the storage space, and there are not enough FreeChunks, it means that the garbage storage space meets the target condition. At this time, the garbage collection process can be triggered, that is, the garbage data index can be obtained to construct a space release request.

[0157] As an optional implementation, the method further includes: in response to the proportion not meeting the proportion threshold, performing a garbage collection operation on the storage space.

[0158] In this embodiment, the proportion of garbage storage space to storage space may be obtained, and when it is determined that the proportion does not meet a proportion threshold, a garbage collection operation needs to be performed on the storage space.

[0159] Optionally, each time a garbage collection operation is performed, that is, each time a Key is deleted or updated, it is necessary to use a simulated pressure device to count and update the garbage ratio, that is, the proportion of garbage storage space to storage space, to evaluate the proportion of garbage space in the current system. When the proportion is greater than or equal to the proportion threshold, the garbage ratio cannot meet the proportion threshold. At this time, it can be said that there is a lot of garbage data in the storage space, and it is necessary to determine the garbage data index corresponding to the garbage data to be pruned, and trigger the corresponding garbage collection operation through the garbage data index.

[0160] Optionally, it is also possible to determine whether it is necessary to start a garbage collection operation by determining the proportion of the target storage space to the storage space. When it is determined that the proportion does not meet a certain proportion threshold, a garbage collection operation can be performed on the storage space. For example, the proportion threshold of the target storage space to the storage space can be pre-set to 95%. After each deletion or update of a Key, the proportion of the target storage space to the storage space of the entire storage device can be obtained in real time through a simulation pressure device. When the proportion is greater than or equal to the proportion threshold, it can be said that the proportion meets the proportion threshold, and garbage collection is not required. When the proportion is less than the proportion threshold, it can be said that the proportion threshold is not met, and garbage collection is required to increase the proportion of the target storage space until it is greater than or equal to 95%, and then garbage collection can be stopped.

[0161] For example, in the embodiment of the present disclosure, it is also possible to determine whether it is necessary to start a garbage collection operation by the ratio between the target storage space and the garbage storage space. The target condition can be pre-set as whether the ratio between the target storage space and the garbage storage space is greater than or equal to the ratio threshold (19:1). After each deletion or update of a Key, the ratio between the target storage space and the garbage storage space can be obtained in real time through a simulation pressure device. When the ratio is greater than 19:1, it can be said that the garbage storage space meets the target condition, and garbage collection is not required at this time. When the ratio is less than 19:1, it can be said that the garbage storage space does not meet the target condition. At this time, it is necessary to determine the garbage data index of the garbage data for which the garbage collection operation is to be performed, and construct a corresponding space release request to trim the garbage data in the storage space and recycle the garbage storage space.

[0162] It should be noted that the above target conditions for determining whether garbage collection operations need to be performed are only examples and are not specifically limited here. As long as the target conditions can trigger the execution of garbage collection operations, they are within the protection scope of the embodiments of the present disclosure.

[0163] As an optional implementation, the method further includes: in response to the garbage storage space not meeting the target condition, performing a garbage collection operation on the storage space.

[0164] In this embodiment, when the garbage storage space does not meet the target condition, a garbage collection operation may be performed on the storage space.

[0165] Optionally, if it is determined that garbage collection is required, the garbage collection process needs to be triggered.

[0166] As an optional embodiment, the storage space includes a target storage space, which is the remaining storage space in the storage space excluding the garbage storage space. The method also includes: when the target data in the data file to be processed is successfully written from the garbage storage space to the target storage space, updating the memory index data result of the simulation pressure device, wherein the target data is the remaining data in the data file to be processed excluding the garbage data, the memory index data structure includes a garbage data index and a target data index of the target data, and the target data index is used to search for the target data in the target storage space.

[0167] In this embodiment, the storage space may include a target storage space, which is the remaining storage space in the storage space excluding the garbage storage space. When the target data in the data file to be processed can be successfully written from the garbage storage space to the target storage space, the memory index data result of the simulation pressure device can be updated, wherein the memory index data structure may include a garbage data index and a target data index of the target data, and the target data index may be used to search for the target data in the target storage space, wherein the memory index data structure may be a memory data structure, such as a ChunkTable.

[0168] Optionally, the in-memory data interface can include a target data index and a non-target data index. The target data index maintains the key of the target data in each chunk, that is, it uniquely identifies the target data in a data file in each corresponding EBS scenario. The non-target data index maintains the location of invalid space in each chunk. By back-checking the target data index, the location of invalid data can also be obtained.

[0169] In the embodiment of the present disclosure, a non-target data index is added, which can more quickly find the invalid space in a Chunk.

[0170] As an optional implementation, the memory index data structure also includes at least one of the following: the position of the garbage data corresponding to the garbage data index in the data block, the position of the target data corresponding to the target data index in the data block, and the target data block list where the data block is located.

[0171] In this embodiment, the memory index data structure may also include at least one of the following: the position of the garbage data in the data block corresponding to the garbage data index, the position of the target data in the data block corresponding to the target data index, and the target data block list where the data block is located.

[0172] Optionally, the Location Table maintains the location of the data corresponding to a Key in the Chunk, that is, the offset and data length of the Key in the Chunk. The FreeChunkList can maintain a list of available Chunks.

[0173] As an optional implementation, the simulated stressor is used to store multiple data blocks maintained by a single-machine storage engine.

[0174] In this embodiment, a plurality of data blocks maintained by a single-machine storage engine may be stored by simulating a stressor.

[0175] In the embodiment of the present disclosure, multiple chunks are maintained in the storage engine, and chunks only support append-writes, and their characteristics are consistent with those of data files in the EBS scenario.

[0176] Optionally, a Chunk may contain target data and garbage space, and the above-mentioned multiple Chunks may be stored in the simulation pressure device.

[0177] The present disclosure also provides a method for verifying a space release operation. FIG5 is a flow chart of a method for verifying a space release operation according to an embodiment of the present disclosure. As shown in FIG5 , the method may include the following steps:

[0178] Step S502: monitor the storage space of the storage server.

[0179] In the technical solution provided in the above step S502 of the present disclosure, the storage space of the storage server can be monitored.

[0180] Optionally, in the process of monitoring the storage space in the storage server through a simulated stressor, the simulated stressor can be used to monitor whether there are deleted or updated data identifiers in the storage server. When a deleted or updated data identifier is detected, it can be determined from the storage space whether there is garbage storage space.

[0181] Step S504 , when it is detected that there is a garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space is obtained, wherein the garbage data index is used to search for garbage data in the to-be-processed data file associated with the load in the garbage storage space.

[0182] In the technical solution provided in the above step S504 of the present disclosure, when it is detected that there is garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space can be obtained, wherein the garbage data index can be used to search for garbage data of the load-associated data files to be processed in the garbage storage space.

[0183] Optionally, by analyzing the current storage space situation in the storage server, it is determined whether the Trim process is triggered.

[0184] For example, if the target condition is to determine whether there are enough FreeChunks in the storage space, and there are not enough FreeChunks, it means that the garbage storage space meets the target condition. At this time, the garbage collection process can be triggered, that is, the garbage data index can be obtained to construct a space release request.

[0185] Step S506: construct a space release request based on the garbage data index.

[0186] In the technical solution provided in the above step S506 of the present disclosure, a space release request may be constructed based on the garbage data index.

[0187] Optionally, a space release request may be constructed based on a garbage data index, that is, the space release request may store a garbage data index corresponding to the garbage data to be released.

[0188] Step S508 : calling an operation interface to respond to the space release request, performing a space release operation on the garbage storage space based on the garbage data index, and releasing the garbage data in the garbage storage space.

[0189] In the technical solution provided in the above step S508 of the present disclosure, after monitoring the space release request of the simulation pressure device, the operation interface can be called to respond to the space release request, and the space release operation can be performed on the garbage storage space based on the garbage data index to release the corresponding garbage data in the garbage storage space.

[0190] Optionally, after detecting that the simulation pressure device has a space release request, the operation interface can be called to filter out garbage data from the data file to be processed by the garbage data index corresponding to the garbage data to be released contained in the space release request, and the garbage data can be released from the garbage storage space where it is located.

[0191] For example, for TrimFile technology, the core implementation is the Trim operation of a stand-alone storage engine, which realizes the deletion function for a section of data. That is, after detecting a space release request, the operation interface can be called to determine the corresponding garbage data for deletion based on the garbage data index in the space release request, thereby releasing the garbage storage space for storing the garbage data.

[0192] Step S510 , in the process of releasing garbage data in the garbage storage space, obtaining a storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space.

[0193] In the technical solution provided in the above step S510 of the present disclosure, when calling the operation interface to respond to the space release request, a release operation is performed on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space. In the process of releasing the garbage data in the garbage storage space, the storage index in the storage space can be obtained.

[0194] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called. The data obtained in the current release process can be compared with the business layer write amplification to verify the function and expected application effect of the Trime technology.

[0195] Step S512: Verify the storage indicator to obtain a verification result.

[0196] In the technical solution provided in the above step S512 of the present disclosure, in the process of releasing the garbage data in the garbage storage space, after obtaining the storage index of the storage space, the storage index can be verified to obtain a verification result.

[0197] Optionally, the storage indicators during the process of releasing the garbage data are verified to determine whether the function of the Trim technology can achieve the expected application effect.

[0198] For example, under the same configuration, you can collect business layer write amplification when Trim is enabled to release garbage data, or you can pre-collect business layer write amplification when Trim is disabled. By comparing the two business layer write amplifications, you can determine whether the Trim technology can achieve the expected results.

[0199] In the disclosed embodiment, the simulated stressor in the storage server is used to simulate the load in the storage service scenario in a stand-alone environment, and the Trim technology of the ZNS SSD storage engine can be applied in the simulated stressor. By comparing the business layer write method, the function and expected application effect of the Trim technology can be verified, thereby improving the verification efficiency of the space release operation. In addition, in this process, the evaluation dependency is streamlined, and the simulated stressor is used in a stand-alone environment based on the functional pressure model to reduce the triggering of garbage collection and reduce the business layer write amplification, thereby achieving the technical effect of reducing the verification cost of the space release operation.

[0200] Through the above steps S502 to S512 of the present invention, the storage space of the storage server is monitored; when it is detected that there is garbage storage space in the storage space, the garbage data index corresponding to the garbage storage space is obtained, wherein the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; a space release request is constructed based on the garbage data index; the space release request is sent to the operation interface of the stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0201] According to an embodiment of the present disclosure, an embodiment of a verification system for space release operations is also provided. Figure 6 is a schematic diagram of a verification system for space release operations according to an embodiment of the present disclosure. As shown in Figure 6, the verification system 600 for space release operations may include: a simulator 601 and a stand-alone storage engine 602.

[0202] The simulator stressor 601 is configured to monitor the storage space; when it is detected that there is garbage storage space in the storage space, the simulator stressor 601 obtains the garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data file in the garbage storage space; and constructs a space release request based on the garbage data index.

[0203] In this embodiment, the storage space can be monitored by the simulation pressure device 601, and when garbage storage space is detected in the storage space, the garbage data index corresponding to the garbage storage space can be obtained, and a space release request can be constructed based on the garbage data index.

[0204] Optionally, the load characteristics of at least the storage service scenario are simulated by the simulator 601 on the storage server, and during the simulation process, the storage space of the memory server can also be monitored in real time by the simulator 601. When junk data is found in the load-related data files to be processed in the monitoring storage space, a corresponding space release request can be constructed based on the corresponding junk data index.

[0205] Optionally, based on EBS IO load analysis, a single-machine stressor can be used to simulate EBS IO load characteristics in a single-machine environment. This simulation can include simulating the cloud disk frontend write and garbage collection flows. You can also add processes to simulate processes such as trim operations using a single-machine storage engine.

[0206] Optionally, after the stress simulator 601 detects a space release request, the space release request may be sent to the stand-alone storage engine 602 .

[0207] The stand-alone storage engine 602 is configured to call an operation interface to respond to a space release request, perform a space release operation on the garbage storage space based on a garbage data index, and release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtain the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; and verify the storage index to obtain a verification result.

[0208] In this embodiment, the operation interface can be called by the stand-alone storage engine 602 to respond to the space release request, and the space release operation can be performed on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space. In the process of releasing the garbage data in the garbage storage space, the storage index of the storage space can be obtained, and the verification result can be obtained by verifying the storage index.

[0209] Optionally, after detecting that the simulation pressure device has a space release request, the operation interface can be called to filter out garbage data from the data file to be processed by the garbage data index corresponding to the garbage data to be released contained in the space release request, and the garbage data can be released from the garbage storage space where it is located.

[0210] Optionally, during the process of releasing the corresponding garbage data in the garbage storage space, the business layer write amplification can be called. The data obtained in the current release process can be compared with the business layer write amplification to verify the function and expected application effect of the Trime technology.

[0211] Optionally, the storage indicators during the process of releasing the garbage data are verified to determine whether the function of the Trim technology can achieve the expected application effect.

[0212] For example, under the same configuration, you can collect business layer write amplification when Trim is enabled to release garbage data, or you can pre-collect business layer write amplification when Trim is disabled. By comparing the two business layer write amplifications, you can determine whether the Trim technology can achieve the expected results.

[0213] In this embodiment, a verification system for space release operations is provided. The storage space is monitored by a simulated pressure device; when it is detected that there is garbage storage space in the storage space, a garbage data index corresponding to the garbage storage space is obtained, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data files in the garbage storage space; a space release request is constructed based on the garbage data index; an operation interface is called by a stand-alone storage engine to respond to the space release request, and a space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, a storage index of the storage space is obtained; the storage index is verified to obtain a verification result, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0214] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure, such as the data for verification, are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0215] Currently, TrimFile is a new interface and capability provided by the Pangu distributed file system based on the ZNS SSD standalone storage engine. It provides upper-level businesses with the ability to release fine-grained space resources within files. Its core technology is the standalone Trim technology provided by Pangu's standalone storage engine, ChunkServer. Compared to the previous practice of using entire files (typically hundreds of MB to several GB in size) as the minimum unit of space resource management, Trim-related technologies break down space resource management units into a specific segment of file data (typically 256KB). The LSM Tree architecture's business layer can release garbage space in a fine-grained manner without the need for frequent garbage collection, thus offering significant application advantages.

[0216] In related technologies, when evaluating TrimFile technology, a full-link evaluation method for storage server scenarios can usually be adopted. This method can usually include steps such as cluster establishment, server deployment, verification scenario construction, and full-link parameter and configuration adjustment. The above method not only has cumbersome evaluation steps and complex coordination, but also relies heavily on hardware and modules for parameter adjustment, resulting in high evaluation costs. Therefore, there is still a technical problem of high verification costs for space release operations.

[0217] Furthermore, the present disclosure provides an evaluation method for the Trim technology of the ZNS SSD stand-alone storage engine, which solves the technical problem of high verification cost of space release operations. It is different from the traditional solution through cluster building, server deployment, verification scenario construction, and full-link parameter and configuration adjustment steps. However, the above method not only has cumbersome evaluation steps and complex coordination, but also relies heavily on hardware and modules for parameter adjustment, resulting in high evaluation costs, and solves the technical problem of high verification cost of space release operations.

[0218] In an embodiment of the present disclosure, in the process of simulating the load in the storage service scenario where the storage server is located by using a simulator, the storage space in the storage server can be monitored in real time by using a simulator. When it is detected that there is garbage storage space in the storage space, the garbage data in the data file to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. The corresponding space release request can be generated according to the garbage data index. In response to the space release request, the corresponding operation interface is called, and the space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data. In the process of releasing garbage data, the storage index of the storage space can be obtained, and the storage index can be verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFlie technology can be simplified through the above steps, the purpose of streamlining the evaluation dependency and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0219] The above method of this embodiment is further introduced below.

[0220] In this embodiment, FIG7 is a schematic diagram of a deployment of a Pangu file according to an embodiment of the present disclosure. As shown in FIG7 , the Pangu distributed file system provides the functional modules with the file semantics of the storage engine link, that is, it only supports appending and does not support overwriting of already written locations. In implementation, a Pangu file is usually divided into data blocks of variable length, such as chunk 1, chunk 2, chunk 3, and chunk 4. Each chunk is stored on different ChunkServers in multiple copies or in a copy-free manner. For example, chunk 2 is stored in the corresponding chunk service A, chunk service B, and chunk service C in the form of copy a, copy b, and copy c.

[0221] Optionally, FIG8 is a schematic diagram of a Pangu TrimFile capability according to an embodiment of the present disclosure. As shown in FIG8 , the Pangu TrimFile capability can be used with the Trim operation of a stand-alone storage engine. Usually, a Pangu file is the smallest unit of resource management, and the entire file needs to be deleted before the space can be released. TrimFile can specify a certain range of data (Range) of a file, for example, data A, and the data A can be deleted without deleting the entire file, that is, the target space can be released. The actual space release operation can be performed by the stand-alone storage engine where the corresponding copy is located, that is, the Trim operation of the ZNS SSD stand-alone storage engine can be called. That is, if it is detected that the space where data A in the Pangu file is located needs to be released, it can be determined on which block the data A is located, for example, on block 5, and it can be further confirmed in which copy of the block the data is located, for example, data A is located on copy d, copy e, and copy f.

[0222] In this embodiment, the demand background of the present disclosure is to reduce the cost of quantitative evaluation of Trim technology. As a new technology provided by Pangu, TrimFile requires a reliable evaluation method to accurately evaluate the functions and technical effects of this technology. The verification and evaluation of this technology is different from the evaluation of the conventional Pangu single functional interface, that is, the evaluation of conventional indicators such as data correctness, latency, IOPS, etc., and relies on the collaborative evaluation of the upper business layer of Pangu, including: functional module architecture transformation to access Trim; adjustment and optimization of garbage collection related strategies of functional modules; construction of targeted business layer garbage collection scenarios to perform functional verification; output of key indicators related to functional modules during the test process (write amplification, garbage collection traffic); comparison of key indicators under the same test scenario for the two conditions of using and not using the Trim function.

[0223] For example, in the EBS scenario, the hardware conditions required for evaluating Trim technology may include storage servers and pressure servers. Five storage servers may be required, and one pressure server may be required.

[0224] Alternatively, Figure 9 is a schematic diagram of a cluster for evaluating Trim effects in a related technology according to an embodiment of the present disclosure. As shown in Figure 9 , the functional modules required for evaluating Trim technology may include at least block services, garbage collection tasks, a master, data block services, and devices. Two services are primarily deployed in this cluster: the Block Storage service and the Pangu service. Trim effect evaluation relies on the coordinated collaboration of at least five functional modules within these two services.

[0225] Optionally, Figure 10 is a schematic diagram of a cluster for evaluating the Trim effect in another related technology according to an embodiment of the present disclosure. As shown in Figure 10, when evaluating the effect of the Trim technology, the main focus is on two key indicators in the block storage service, write amplification and garbage collection write volume. It is also necessary to pay attention to two key statistics, the amount of data written to the cloud disk and the amount of data written by garbage collection, among which the amount of data written to the cloud disk (foreground write traffic) is the actual amount of data written by the cloud disk user, which is counted by the EBS BlockServer module. The amount of data written by garbage collection is the amount of data that is read in the process of recycling the garbage data of the cloud disk and written to the new file, which is counted by the EBS GCWorker module.

[0226] Optionally, storage indicator: EBS write amplification = (data volume written to the cloud disk + data volume written by garbage collection) / data volume written to the cloud disk.

[0227] Optionally, Figure 11 is a schematic diagram of the evaluation results of a TrimFile technology in a full-link test according to an embodiment of the present disclosure. As shown in Figure 11, based on the above method, a comparison of the actual evaluation results can be performed, wherein the evaluation result can be a normalized result of 1M random write. Applying the above evaluation method, the write amplification and garbage collection write volume before and after using Pangu TrimFile technology are compared, thereby effectively proving that the application of Trim technology has a good effect on reducing business layer write amplification and reducing garbage collection traffic. For example, when TrimFile is closed, the write amplification can be 1 and the garbage collection write volume can be 1. However, when TrimFile is opened, the write amplification can be 0.64 and the garbage collection write volume can be 0.53.

[0228] However, the evaluation method described above is costly and complex. A full-link evaluation approach can accurately assess the technical effectiveness of TrimFile technology at the business level. However, the hardware requirements and functional module dependencies are relatively high. For example, a physical cluster requires five storage servers, 40 ZNS SSDs, and two service deployments, such as the EBS service and the Pangu service. Five key functional modules are also required, such as the EBS BlockServer, EBS GC Worker, Pangu Master, Pangu Worker, and Pangu ChunkServer. An EBS stressor, such as VolumeBenchmark, is also required. Multiple key parameters must be adjusted, such as those related to EBS garbage collection triggering and rate limiting. Key metrics, such as the amount of data written to the cloud disk and the amount of garbage collected data, must also be collected. Therefore, based on the above analysis, the entire evaluation solution has high hardware costs, multiple module dependencies, and complex parameter adjustments. Consequently, the high cost of verifying space-release operations remains a technical challenge.

[0229] In this disclosed embodiment, considering the above issues, a Trim technology evaluation method based on the ZNS SSD standalone storage engine was designed. Pangu's new TrimFile technology core implementation is the Trim operation of the standalone storage engine, enabling the deletion of a segment of data. Therefore, this disclosed embodiment shortens the verification process for technical effectiveness and focuses the evaluation on the Trim operation of the standalone storage engine.

[0230] Optionally, Figure 12 is a schematic diagram of an evaluation system for Trim technology according to an embodiment of the present disclosure. As shown in Figure 12, the implementation idea of ​​the present disclosure is to simulate the IO load characteristics of EBS in a stand-alone environment based on the IO load analysis of the EBS functional module, including the cloud disk foreground write and garbage collection processes, and add a new process access using the Trim operation of the stand-alone storage engine to reduce the triggering of garbage collection and reduce the write amplification at the business level, thereby achieving the purpose of verifying the effect of the Trim technology, and further achieving the technical effect of reducing the cost of verifying the space release operation. As shown in Figure 12, the embodiment of the present disclosure simplifies the evaluation dependencies, uses a stand-alone stressor, and simulates the foreground write traffic, garbage collection read and write traffic, and Trim traffic in a stand-alone environment based on the functional pressure model. Based on the above design, the evaluation process is greatly simplified in terms of multiple dependencies on clusters, EBS services, and Pangu services. Only one ZNS SSD server is needed to run the evaluation test of the TrimFile effect.

[0231] Optionally, Figure 13 is a schematic diagram of a functional fuzzy stressor module according to an embodiment of the present disclosure. As shown in Figure 13, the overall module design of the functional simulation stressor module can be composed of a memory index data structure and a storage engine data file. The chunk table (ChunkTable) in the memory data structure may include a target data index and a garbage data index, wherein the target data index may maintain the key of the target data in each Chunk (corresponding to a data file in the EBS scenario). The garbage data index may maintain the invalid space position in each Chunk (each invalid space is identified by an offset and a length, i.e., Location). The location of the invalid data can also be obtained by reverse querying the target data index. The embodiment of the present disclosure adds a garbage data index, which can more quickly find the invalid space in a Chunk, i.e., the garbage storage space. The memory data structure may also include a LocationTable, which can be used to maintain the position of the data corresponding to a Key in the Chunk, i.e., the offset and data length in the Chunk. The memory data structure may also include a FreeChunkList, which can be used to maintain a list of available Chunks.

[0232] Optionally, as shown in Figure 13, the storage engine data file maintains multiple chunks. Chunks only support append-only writes, and their characteristics are consistent with those of data files in EBS scenarios. Chunks contain target data and garbage data associated with garbage data indexes in the garbage space.

[0233] Optionally, the key operation process of the embodiment of the present disclosure may include a write process. In the write process, a Chunk may contain multiple data uniquely identified by a Key, and the data (Value) is written to the Chunk in append mode through Put(Key, Value), and the ChunkTable and LocationTable in memory are updated.

[0234] Optionally, the key operation process of the embodiment of the present disclosure may also include an update or delete process. In the update or delete process, if an update or delete operation is performed on a Key, the ChunkTable in memory may be updated, the Key may be updated or deleted from the target data index and LocationTable, and the corresponding data space may be added to the invalid data index, that is, the garbage data index.

[0235] Optionally, FIG14 is a schematic diagram of a garbage collection method using the Trim technology according to an embodiment of the present disclosure. As shown in FIG14 , the method may include the following steps:

[0236] Step S1401, delete or update the Key.

[0237] In this embodiment, the Key can be deleted or updated.

[0238] Optionally, when a Key is deleted or updated, it may be indicated that garbage storage space is generated, and step S1402 may be further executed.

[0239] Step S1402: Update the garbage ratio.

[0240] In this embodiment, the garbage space ratio in the current system may be evaluated by counting and updating the garbage ratio, and step S1403 may be further executed.

[0241] Step S1403: determine whether the Trim process is triggered.

[0242] In this embodiment, it can be determined whether the Trim process is triggered. If so, step S1406 can be executed; otherwise, step S1404 can be executed.

[0243] Optionally, by analyzing the current space situation, it is determined whether the Trim process has been triggered. If it is determined that garbage collection is required, for example, if FreeChunk is insufficient, the garbage collection process needs to be triggered.

[0244] Step S1404: triggering garbage collection.

[0245] In this embodiment, garbage collection may be triggered.

[0246] Optionally, in the garbage collection process, the target data in a Chunk can be read from the old location and all written to the new location, and the old Chunk can be deleted to release the storage space occupied by the garbage.

[0247] Step S1405: Update write amplification.

[0248] In this embodiment, the update may be performed with write amplification after a garbage collection operation.

[0249] Optionally, the write amplification information may also be statistically analyzed periodically, without necessarily relying on the garbage collection process.

[0250] Step S1406 , scanning the garbage data index in the chunk table to obtain garbage space information.

[0251] In this embodiment, if the Trim process is triggered, the ChunkTable garbage data index can be scanned to obtain the corresponding garbage space information.

[0252] Optionally, when the Trim triggering condition is reached, the stressor scans the garbage data index in the ChunkTable and constructs a Trim request.

[0253] Step S1407: call the storage engine Trim operation to release garbage space.

[0254] In this embodiment, the storage engine Trim operation may be called to release garbage space.

[0255] Optionally, call the Trim interface of the storage engine and send a Trim request to the storage engine. This releases garbage space by calling the stand-alone storage engine.

[0256] Step S1408: The space release operation is successful, and new available chunks are created as needed to support more writing.

[0257] In this embodiment, it can be determined whether the space release operation is successful. If the space is released successfully, a new available block can be created as needed to support more writing.

[0258] Optionally, you can call the standalone storage engine's storage information interface to confirm whether the space release is successful. After the space is released successfully, you can support new writes based on the remaining space and remaining available chunks.

[0259] In the disclosed embodiments, by applying the aforementioned evaluation method, the reliance on the Trim technology verification can be reduced as follows: the number of ZNS SSD servers is reduced by 80%, with only one ZNS SSD server required; the number of ZNS SSD disks is reduced from 60 to 1, and from 60 disks in the entire cluster to a maximum of one ZNS SSD disk; the memory resources are reduced from 500GB+ to 2GB, and from a total of 500GB+ memory resources in the entire cluster to 2GB memory resources, without relying on the deployment of EBS and Pangu services.

[0260] Optionally, Figure 15 is a schematic diagram illustrating the results of a Trim validation according to an embodiment of the present disclosure. As shown in Figure 15 , the technical effect of the embodiment of the present disclosure is that, in scenarios where the conditions are met, garbage collection at the business layer can be significantly reduced, and write amplification can be maintained at an ideal level. The first row of Figure 15 shows the validation parameters. This validation simulated 300 files, each 10MB in size, at an 80% watermark, with 10% simulating cloud disk reserved space. The second row shows the key statistical results. In a steady-state state, write amplification can be maintained at 1.08, which is a scenario that meets the conditions. This means that each garbage space can be directly released through Trim technology. In actual use cases, there are more requirements for garbage space continuity, and the actual effect depends on the specific pattern of the functional module. The garbage ratio (Stales) can be maintained below 5%, with a typical value of 15% in use cases. The number of Trim requests (Trims) is the number of Trim requests executed within a 5-second window. The actual occupied physical space (PhyData) is the actual occupied physical space within the storage engine.

[0261] Optionally, Figure 16 is a schematic diagram of a comparison of write amplification with Trim turned off and on according to an embodiment of the present disclosure. As shown in Figure 16, under the same configuration, the measured write amplification comparison is shown in Figure 16 for two cases where Trim is turned on and off. That is, when Trim is turned on, the write amplification obtained using the method of the embodiment of the present disclosure is 0.28, and when Trim is turned off, the write amplification obtained is 1.

[0262] In the disclosed embodiments, the single-machine environment simulation method can be used to evaluate the expected effects of the Trim technology. In the functional simulation stressor, the Trim technology of the ZNS SSD storage engine can be applied to verify the function and expected effects of the Trim technology by comparing the business layer amplification. Through the above method, the evaluation cost can be significantly reduced by at least 80%.

[0263] In an embodiment of the present disclosure, in the process of simulating the load in the storage service scenario where the storage server is located by using a simulator, the storage space in the storage server can be monitored in real time by using a simulator. When it is detected that there is garbage storage space in the storage space, the garbage data in the data file to be processed associated with the load can be found in the garbage storage space, and the garbage data index of the garbage storage space can be obtained. The corresponding space release request can be generated according to the garbage data index. In response to the space release request, the corresponding operation interface is called, and the space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data. In the process of releasing garbage data, the storage index of the storage space can be obtained, and the storage index can be verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated through the verification result. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFlie technology can be simplified through the above steps, the purpose of streamlining the evaluation dependency and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0264] According to an embodiment of the present disclosure, a space release operation verification device for implementing the space release operation verification method shown in FIG. 3 is also provided.

[0265] Figure 17 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present disclosure. As shown in Figure 17, the verification device 1700 for the space release operation may include: a first monitoring unit 1702, a first calling unit 1704, a first acquisition unit 1706 and a first verification unit 1708.

[0266] The first monitoring unit 1702 is configured to monitor the space release request of the simulation stressor, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation stressor detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space.

[0267] The first calling unit 1704 is configured to call the operation interface of the stand-alone storage engine to respond to the space release request, and perform a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space.

[0268] The first acquiring unit 1706 is configured to acquire a storage index of the storage space during the process of releasing garbage data in the garbage storage space, wherein the storage index is used to represent the performance of writing data to the storage space.

[0269] The first verification unit 1708 is configured to verify the storage indicator and obtain a verification result.

[0270] Here, the first monitoring unit 1702, the first calling unit 1704, the first obtaining unit 1706, and the first verification unit 1708 correspond to steps S302 to S308 in Example 1. The examples and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b..., 102n). The above-mentioned units can also be part of the device and can be run in the computer terminal 10 provided in Example 7.

[0271] According to an embodiment of the present disclosure, a space release operation verification device for implementing the space release operation verification method shown in FIG. 4 is also provided.

[0272] Figure 18 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present disclosure. As shown in Figure 18, the verification device 1800 for the space release operation may include: a second monitoring unit 1802, a second acquisition unit 1804, a first construction unit 1806 and a first sending unit 1808.

[0273] The second monitoring unit 1802 is configured to monitor the storage space of the storage server.

[0274] The second obtaining unit 1804 is configured to obtain a garbage data index corresponding to the garbage storage space when detecting the presence of garbage storage space in the storage space, wherein the garbage data index is used to search for garbage data in the to-be-processed data file associated with the load in the garbage storage space.

[0275] The first constructing unit 1806 is configured to construct a space release request based on the garbage data index.

[0276] The first sending unit 1808 is configured to send a space release request to the operation interface of the stand-alone storage engine, wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0277] It should be noted that the second monitoring unit 1802, the second acquisition unit 1804, the first construction unit 1806, and the first sending unit 1808 correspond to steps S402 to S408 in Example 1. The examples and application scenarios implemented by the four units and the corresponding steps are the same, but are not limited to the contents disclosed in Example 1. It should be noted that the above-mentioned units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b..., 102n). The above-mentioned units can also be part of the device and can be run in the computer terminal 10 provided in Example 7.

[0278] According to an embodiment of the present disclosure, a space release operation verification device for implementing the space release operation verification method shown in FIG. 5 is also provided.

[0279] Figure 19 is a schematic diagram of a verification device for a space release operation according to an embodiment of the present disclosure. As shown in Figure 19, the verification device 1900 for the space release operation may include: a third monitoring unit 1902, a third acquisition unit 1904, a second construction unit 1906, a second calling unit 1908, a third acquisition unit 1910 and a second verification unit 1912.

[0280] The third monitoring unit 1902 is configured to monitor the storage space of the storage server.

[0281] The third acquisition unit 1904 is configured to acquire a garbage data index corresponding to the garbage storage space when detecting the presence of garbage storage space in the storage space, wherein the garbage data index is used to search for garbage data in the to-be-processed data file associated with the load in the garbage storage space.

[0282] The second constructing unit 1906 is configured to construct a space release request based on the garbage data index.

[0283] The second calling unit 1908 is configured to call the operation interface to respond to the space release request, and perform a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space.

[0284] The third acquiring unit 1910 is configured to acquire a storage index of the storage space during the process of releasing garbage data in the garbage storage space, wherein the storage index is used to represent the performance of writing data to the storage space.

[0285] The second verification unit 1912 is configured to verify the storage indicator and obtain a verification result.

[0286] It should be noted that the third monitoring unit 1902, the third acquisition unit 1904, the second construction unit 1906, the second calling unit 1908, the third acquisition unit 1910, and the second verification unit 1912 correspond to steps S502 to S512 in Example 1. The six units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Example 1. It should be noted that the above-mentioned units can be hardware components or software components stored in a memory (e.g., memory 2104) and processed by one or more processors (e.g., processors 2102a, 2102b..., 2102n). The above-mentioned units can also be part of the device and can be run in the computer terminal 210 provided in Example 7.

[0287] In the verification device for the space release operation, in the process of simulating the load in the storage service scenario in which the storage server is located by using a simulated stressor, the storage space in the storage server can be monitored in real time by using the simulated stressor. When garbage storage space is detected in the storage space, garbage data in the to-be-processed data file associated with the load can be found in the garbage storage space, and a garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated based on the garbage data index. In response to the space release request, the corresponding operation interface is called, and based on the garbage data index, a space release operation is performed on the garbage storage space to release the garbage data. In the process of releasing the garbage data, the storage index of the storage space can be obtained and verified to obtain a verification result. The function and service effect of the TrimFile technology can be evaluated based on the verification result. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependency and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0288] The embodiment of the present disclosure may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0289] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of a computer network.

[0290] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the verification method of the space release operation: monitoring the space release request of the simulated pressure device, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulated pressure device detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; calling the operation interface of the stand-alone storage engine to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verifying the storage index to obtain a verification result.

[0291] Alternatively, Figure 20 is a block diagram of a computer terminal according to an embodiment of the present disclosure. As shown in Figure 20 , the computer terminal A may include: one or more (only one is shown in the figure) processors 2002 , a memory 2004 , and a transmission device 2006 .

[0292] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the verification method and device for space release operation in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned verification method for space release operation. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0293] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: call the operation interface to respond to the space release request, find the initial storage location of the garbage data in the garbage storage space based on the garbage data index; and delete the garbage data at the initial storage location in the garbage storage space.

[0294] Optionally, the processor may also execute program code for the following steps: determining a data block including junk data in a junk storage space; identifying target data in a data file to be processed in the data block, wherein the target data is the remaining data other than the junk data in the data file to be processed; reading the target data from the data block; and when the read target data is successfully written to the target storage space, deleting the data block including the junk data at the initial storage location to delete the junk data.

[0295] Optionally, the processor may further execute program code of the following steps: detecting a first written data volume of target data written to the target storage space during the process of releasing garbage data in the garbage storage space; and determining a storage index based on the first written data volume.

[0296] Optionally, the processor may further execute program code of the following steps: detecting a second amount of written data obtained by performing a write operation on the storage space during the process of releasing garbage data in the garbage storage space; and determining a storage index based on the first amount of written data and the second amount of written data.

[0297] Optionally, the processor may also execute the program code of the following steps: in response to a shutdown instruction, obtaining the initial storage indicator of the storage space in the storage service scenario, wherein the shutdown instruction is used to indicate prohibition of calling the operation interface; and verifying the storage indicator using the initial storage indicator to obtain a verification result.

[0298] Optionally, the processor may further execute program code of the following steps: upon successfully releasing garbage data in the garbage storage space, updating the target storage space in the storage space based on the released garbage storage space; and calling an information interface to perform a write operation in the updated target storage space.

[0299] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: monitor the storage space of the storage server; when it is detected that there is garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data files in the garbage storage space; construct a space release request based on the garbage data index; send the space release request to the operation interface of the stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0300] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: monitor the storage space of the storage server; when it is detected that there is garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated to-be-processed data files in the garbage storage space; construct a space release request based on the garbage data index; call the operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtain the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verify the storage index to obtain a verification result.

[0301] According to an embodiment of the present disclosure, a method for verifying a space release operation is provided. In the embodiment of the present disclosure, in the process of simulating the load in the storage service scenario in which the storage server is located by using a simulated stressor, the storage space in the storage server can be monitored in real time by using the simulated stressor. When garbage storage space is detected in the storage space, garbage data in the to-be-processed data file associated with the load can be found in the garbage storage space, and a garbage data index of the garbage storage space can be obtained. A corresponding space release request can be generated based on the garbage data index. In response to the space release request, the operation interface of the stand-alone storage engine is called, and a space release operation is performed on the garbage storage space based on the garbage data index to release the garbage data. During the process of releasing the garbage data, the storage index of the storage space can be obtained and verified to obtain a verification result. The verification result can be used to evaluate the function and service effect of the TrimFile technology. Since the embodiment of the present disclosure takes into account that the evaluation process of the TrimFile technology can be simplified through the above steps, the purpose of streamlining the evaluation dependency and shortening the verification link of the effect of the technology is achieved, thereby achieving the technical effect of reducing the verification cost of the space release operation and solving the technical problem of high verification cost of the space release operation.

[0302] Those skilled in the art will appreciate that the structure shown in FIG20 is merely illustrative, and that computer terminal A may also be a smartphone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG20 does not limit the structure of the aforementioned computer terminal A. For example, computer terminal A may include more or fewer components (e.g., a network interface, a display device, etc.) than those shown in FIG20 , or may have a configuration different from that shown in FIG20 .

[0303] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0304] The embodiment of the present disclosure further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the space release operation verification method provided in the first embodiment.

[0305] Computer-readable storage media may also be referred to as computer storage media. They may include data signals transmitted in baseband or as part of a carrier wave, carrying readable program code. Such transmitted data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable storage media may transmit, propagate, or transfer programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0306] The program code contained in the computer-readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, radio frequency, etc., or any suitable combination of the foregoing.

[0307] Optionally, in this embodiment, the computer-readable storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0308] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for performing the following steps: monitoring a space release request of a simulated stressor, wherein the space release request is constructed based on a garbage data index corresponding to the garbage storage space when the simulated stressor detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; calling the operation interface of a stand-alone storage engine to respond to the space release request, and performing a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining a storage indicator of the storage space, wherein the storage indicator is used to represent the performance of writing data to the storage space; and verifying the storage indicator to obtain a verification result.

[0309] Optionally, the computer-readable storage medium may also execute program code for the following steps: calling an operation interface to respond to a space release request, searching for the initial storage location of the garbage data in the garbage storage space based on a garbage data index; and deleting the garbage data at the initial storage location in the garbage storage space.

[0310] Optionally, the computer-readable storage medium may also execute program code for the following steps: determining a data block including junk data in a junk storage space; identifying target data in a data file to be processed in the data block, wherein the target data is the remaining data other than the junk data in the data file to be processed; reading the target data from the data block; and when the read target data is successfully written to the target storage space, deleting the data block including the junk data at the initial storage location to delete the junk data.

[0311] Optionally, the computer-readable storage medium may further execute program code for the following steps: detecting a first written data volume of target data written to the target storage space during the process of releasing garbage data in the garbage storage space; and determining a storage index based on the first written data volume.

[0312] Optionally, the computer-readable storage medium may also execute program code for the following steps: detecting a second amount of written data obtained by performing a write operation on the storage space during the process of releasing garbage data in the garbage storage space; and determining a storage index based on the first amount of written data and the second amount of written data.

[0313] Optionally, the above-mentioned computer-readable storage medium can also execute the program code of the following steps: in response to a shutdown instruction, obtain the initial storage indicator of the storage space in the storage service scenario, wherein the shutdown instruction is used to indicate that calling the operation interface is prohibited; use the initial storage indicator to verify the storage indicator to obtain a verification result.

[0314] Optionally, the computer-readable storage medium may further execute program code for the following steps: upon successfully releasing garbage data in the garbage storage space, updating the target storage space in the storage space based on the released garbage storage space; and calling an information interface to perform a write operation on the updated target storage space.

[0315] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: monitoring the storage space of a storage server; when detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in load-associated to-be-processed data files in the garbage storage space; constructing a space release request based on the garbage data index; sending the space release request to the operation interface of a stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0316] As an optional example, a computer-readable storage medium is configured to store program code for performing the following steps: monitoring the storage space of a storage server; when detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in load-associated to-be-processed data files in the garbage storage space; constructing a space release request based on the garbage data index; calling an operation interface to respond to the space release request, and performing a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining a storage indicator of the storage space, wherein the storage indicator is used to represent the performance of writing data to the storage space; and verifying the storage indicator to obtain a verification result.

[0317] An embodiment of the present disclosure may provide an electronic device, which may include a memory and a processor.

[0318] Figure 21 is a block diagram of an electronic device according to a method for verifying a space release operation according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0319] As shown in Figure 21, the device 2100 includes a computing unit 2101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 2102 or a computer program loaded from a storage unit 2108 into a random access memory (RAM) 2103. Various programs and data required for the operation of the device 2100 can also be stored in the RAM 2103. The computing unit 2101, the ROM 2102, and the RAM 2103 are connected to each other via a bus 2104. An input / output (I / O) interface 2105 is also connected to the bus 2104.

[0320] Various components in device 2100 are connected to I / O interface 2105, including: input unit 2106, such as a keyboard, mouse, etc.; output unit 2104, such as various types of displays, speakers, etc.; storage unit 2108, such as a magnetic disk, optical disk, etc.; and communication unit 2109, such as a network card, modem, wireless communication transceiver, etc. Communication unit 2109 allows device 2100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0321] The computing unit 2101 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 2101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 2101 performs the various methods and processes described above, such as the verification method of the space release operation. For example, in some embodiments, the verification method of the space release operation can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 2108. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 2100 via the ROM 2102 and / or the communication unit 2109. When the computer program is loaded into the RAM 2103 and executed by the computing unit 2101, one or more steps of the verification method of the space release operation described above can be performed. Alternatively, in other embodiments, the computing unit 2101 may be configured to perform the verification method of the space release operation in any other appropriate manner (for example, by means of firmware).

[0322] The embodiment of the present disclosure further provides a chip. Optionally, in this embodiment, the chip may include a processor configured to call and execute a computer program from a memory, so that a device equipped with the chip executes the space release operation verification method of the embodiment of the present disclosure.

[0323] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0324] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0325] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0326] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0327] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0328] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) for displaying information to the user, a monitor; and a keyboard and pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0329] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0330] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0331] The present disclosure also provides a processor. Figure 22 shows a block diagram of a processor for implementing the present disclosure. As shown in Figure 22, the processor 2200 is configured to run a program, wherein the program, when executed by the processor, executes the method in the above embodiment.

[0332] In an embodiment of the present disclosure, the processor 2200 may execute a program for a verification method for a space release operation.

[0333] An embodiment of the present disclosure also provides a computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned image generation method or image generation data processing method.

[0334] The embodiments of the present disclosure further provide a computer program, which, when executed by a processor, implements the steps of the above-mentioned image generation method or image generation data processing method.

[0335] Optionally, the above-mentioned computer program implements program code for the following steps when executed by the processor: monitoring the space release request of the simulation pressure device, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; calling the operation interface of the stand-alone storage engine to respond to the space release request, and based on the garbage data index, performing a space release operation on the garbage storage space to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verifying the storage index to obtain a verification result.

[0336] Optionally, the computer program implements the following program code when executed by the processor: monitoring the storage space of the storage server; when detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; constructing a space release request based on the garbage data index; sending the space release request to the operation interface of the stand-alone storage engine; wherein the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the stand-alone storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

[0337] Optionally, the computer program implements the following program code when executed by the processor: monitoring the storage space of the storage server; when detecting the presence of garbage storage space in the storage space, obtaining a garbage data index corresponding to the garbage storage space, wherein the garbage data index is used to search for garbage data in the load-associated data files to be processed in the garbage storage space; constructing a space release request based on the garbage data index; calling an operation interface to respond to the space release request, and performing a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining a storage indicator of the storage space, wherein the storage indicator is used to represent the performance of writing data to the storage space; and verifying the storage indicator to obtain a verification result.

[0338] It should be noted that the serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

[0339] In the above embodiments of the present disclosure, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0340] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0341] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0342] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0343] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory, a random access memory, a mobile hard disk, a magnetic disk or an optical disk.

[0344] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as within the scope of protection of the present disclosure. Industrial Applicability

[0345] The solution provided by the embodiment of the present disclosure can be applied to the process of verifying the space release operation, monitoring the space release request of the simulation pressure device, wherein the space release request is constructed based on the garbage data index corresponding to the garbage storage space when the simulation pressure device detects that there is garbage storage space in the storage space of the storage server, and the garbage data index is used to search for garbage data in the load-associated to-be-processed data files in the garbage storage space; calling the operation interface of the stand-alone storage engine to respond to the space release request, and performing a space release operation on the garbage storage space based on the garbage data index to release the garbage data in the garbage storage space; in the process of releasing the garbage data in the garbage storage space, obtaining the storage index of the storage space, wherein the storage index is used to represent the performance of writing data to the storage space; verifying the storage index to obtain a verification result, thereby solving the technical problem of high verification cost of the space release operation.< / location> < / location>

Claims

1. A verification method for space release operations, which is applied to a single-machine storage engine deployed on a storage server. A simulation pressure device is also deployed on the storage server, and the simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located. The method includes: Monitoring the space release requests of the simulation pressure device. Among them, the space release requests are constructed based on the garbage data indexes corresponding to the garbage storage spaces when the simulation pressure device detects that there are garbage storage spaces in the storage space of the storage server. The garbage data indexes are used to find the garbage data in the data files to be processed associated with the load in the garbage storage spaces; Invoking the operation interface of the single-machine storage engine to respond to the space release requests, and based on the garbage data indexes, performing space release operations on the garbage storage spaces to release the garbage data in the garbage storage spaces; During the process of releasing the garbage data in the garbage storage spaces, obtaining the storage metrics of the storage space. Among them, the storage metrics are used to represent the performance of writing data to the storage space; Verifying the storage metrics to obtain a verification result.

2. The method according to claim 1, wherein, Invoking the operation interface of the single-machine storage engine to respond to the space release requests, and based on the garbage data indexes, performing space release operations on the garbage storage spaces to release the garbage data in the garbage storage spaces, including: Invoking the operation interface to respond to the space release requests, and in the garbage storage spaces, finding the initial storage locations where the garbage data is located based on the garbage data indexes; Deleting the garbage data at the initial storage locations in the garbage storage spaces.

3. The method according to claim 2, wherein The storage space includes a target storage space, and the target storage space is the remaining storage space in the storage space except the garbage storage spaces. Among them, deleting the garbage data at the initial storage locations in the garbage storage spaces includes: Determining the data blocks in the garbage storage spaces that include the garbage data; Identifying the target data in the data files to be processed in the data blocks. Among them, the target data is the remaining data in the data files to be processed except the garbage data; Reading the target data out of the data blocks; When the read target data is successfully written into the target storage space, deleting the data blocks including the garbage data at the initial storage locations to delete the garbage data.

4. The method according to claim 3, wherein During the process of releasing the garbage data in the garbage storage spaces, obtaining the storage metrics, including: During the process of releasing the garbage data in the garbage storage spaces, detecting the first amount of written data of the target data written into the target storage space; Determining the storage metrics based on the first amount of written data.

5. The method according to claim 4, wherein, The method further includes: During the process of releasing the garbage data in the garbage storage spaces, detecting the second amount of written data obtained by performing a write operation on the storage space; Determine the storage metric based on the first write data volume, including: determining the storage metric based on the first write data volume and the second write data volume.

6. The method according to claim 1, wherein Verify the storage metric to obtain a verification result, including: In response to a close instruction, obtain an initial storage metric of the storage space in the storage service scenario, where the close instruction is used to indicate prohibiting the invocation of the operation interface; Verify the storage metric using the initial storage metric to obtain the verification result.

7. The method according to any one of claims 1 to 6, wherein The method further includes: In the case of successfully releasing the garbage data in the garbage storage space, update the target storage space in the storage space based on the released garbage storage space; Invoke an information interface to perform a write operation on the updated target storage space.

8. A verification method for a space release operation, applied to a simulation pressure device deployed on a storage server, where the simulation pressure device is used to simulate the load in the storage service scenario where the storage server is located, and a single-machine storage engine is also deployed on the storage server. The method includes: Monitor the storage space of the storage server; In the case of detecting a garbage storage space in the storage space, obtain a garbage data index corresponding to the garbage storage space, where the garbage data index is used to find garbage data in the data file to be processed associated with the load in the garbage storage space; Construct a space release request based on the garbage data index; Send the space release request to the operation interface of the single-machine storage engine; Wherein, the space release request is responded to by the operation interface, and the garbage data index in the space release request is used to enable the single-machine storage engine to perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space.

9. The method according to claim 8, wherein The method further includes: In the case of the existence of a data identifier for deletion or update in the simulation pressure device, determine that there is a garbage storage space in the storage space, where the data identifier is used to represent the data stored in the storage space.

10. The method according to claim 8, wherein, Obtain the garbage data index corresponding to the garbage storage space, including: Obtain the proportion of the garbage storage space in the storage space; In response to the proportion not meeting the proportion threshold, obtain the garbage data index corresponding to the garbage storage space.

11. The method according to claim 10, wherein, The method further includes: In response to the proportion not meeting the proportion threshold, perform a garbage collection operation on the storage space.

12. The method according to claim 8, wherein, The storage space includes a target storage space, and the target storage space is the remaining storage space in the storage space except the garbage storage space. The method further includes: When the target data in the data file to be processed is successfully written from the garbage storage space to the target storage space, update the memory index data structure of the simulation pressure device, where the target data is the remaining data in the data file to be processed except the garbage data, and the memory index data structure includes a garbage data index and a target data index of the target data, and the target data index is used to find the target data in the target storage space.

13. The method according to claim 12, wherein, The memory index data structure further includes at least one of the following: the position of the garbage data corresponding to the garbage data index in the data block where it is located, the position of the target data corresponding to the target data index in the data block where it is located, and the target data block list where the data block is located.

14. The method according to claim 8, wherein, The simulation pressure device is used to store multiple data blocks maintained by the single-machine storage engine.

15. A verification method for a space release operation, applied to a storage server, the method includes: Monitor the storage space of the storage server; When it is detected that there is a garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data file to be processed associated with the load in the garbage storage space; Construct a space release request based on the garbage data index; Call an operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; During the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space; Verify the storage metrics to obtain a verification result.

16. A verification system for a space release operation, including: A simulation pressure device configured to monitor the storage space; When it is detected that there is a garbage storage space in the storage space, obtain the garbage data index corresponding to the garbage storage space, where the garbage data index is used to find the garbage data in the data file to be processed associated with the load in the garbage storage space; construct a space release request based on the garbage data index; A single-machine storage engine configured to call an operation interface to respond to the space release request, and based on the garbage data index, perform a space release operation on the garbage storage space to release the garbage data in the garbage storage space; during the process of releasing the garbage data in the garbage storage space, obtain the storage metrics of the storage space, where the storage metrics are used to represent the performance of writing data to the storage space; verify the storage metrics to obtain a verification result.

17. An electronic device, including: A memory storing an executable program; A processor for running the program, where when the program runs, it executes the method according to any one of claims 1 to 15.

18. A chip, comprising: A processor, configured to call and run a computer program from a memory, such that a device installed with the chip executes the method according to any one of claims 1 to 15.

19. A computer program product, comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 15.

20. A computer program, wherein, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 15.

Images