Data processing method and device, computer device, storage medium and program product

Abstract

The application relates to a data processing method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: receiving a data processing request, and performing data processing on a cache disk and a rear-end disk according to corresponding key information and / or a logical space of the data processing request, wherein the logical space corresponds to a data block area in the cache disk, and the key information comprises a mapping relationship between the cache disk and the rear-end disk. The method directly operates the cache disk through the operation on the logical space, does not use its own data block area mapping and space recycling mechanism, can independently complete space allocation, release and mapping, can allocate corresponding matching data block areas by using the association relationship between the logical space and the data block area, can greatly reduce read-write amplification and required reserved space caused by a large number of data block areas, and can further improve the data processing performance of a storage system.

Description

Technical Field

[0001] This application relates to the field of data storage technology, and in particular to a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] Most current storage systems use SSD caching as an acceleration method, but the use of SSDs largely depends on the SSD's own space scheduling, that is, directly providing data and address for writing, and providing address and size for reading. For example, when an SSD is used directly, there is an internal FTL address mapping to map logical addresses to physical addresses, a GC garbage collection mechanism to reclaim block space, and the use of OP space to perform data movement.

[0003] However, while current storage products have achieved the goal of increasing read and write throughput through SSDs, performance and hardware utilization efficiency can still be improved. Therefore, how to optimize the SSD caching mechanism has become a technical problem that data storage systems urgently need to solve. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, storage medium, and program product that can improve SSD caching performance in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a data processing method, including:

[0006] Receive data processing requests;

[0007] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk and a backend disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0008] The method described in the above embodiments proposes the concept of logical space and directly manipulates the cache disk through operations on the logical space without using its own data block area mapping and space reclamation mechanisms. It can autonomously complete space allocation, release, and mapping. Furthermore, it can directly write to the cache disk using sequential append-only writes based on operations on the logical space, and allocate corresponding matching data block areas by utilizing the association between the logical space and data block areas. This significantly reduces read / write amplification and the required reserved space caused by the need for a large number of data block areas, thereby improving the data processing performance of the storage system. In addition, the above method introduces key information corresponding to data processing requests as an index and identifier for each data entry, recording the correspondence between the actual written data and the data on the cache disk. This replaces the original FTL translation layer and the inherent data mapping relationship in the cache disk, reducing FTL maintenance, greatly reducing the additional overhead on the cache disk space, and improving the space utilization of the storage system.

[0009] In one embodiment, the data processing request is the read request, and the data processing based on the key information and logical space corresponding to the data processing request, through the cache disk and the backend disk, includes:

[0010] Based on the address index carried in the read request, the key information is retrieved from the key information record in memory to obtain the retrieval result;

[0011] Data is read from the cache disk and backend disk based on the search results.

[0012] The above embodiments involve a data reading process, and the location of the data to be read is determined by retrieving key information corresponding to the read request. This method differs from the existing storage products that determine the data reading location after mapping based on the FTL translation layer. This method is simple, does not require additional space overhead, and can greatly improve the efficiency of data reading.

[0013] In one embodiment, the step of reading data through a cache disk and a backend disk based on the retrieval results includes:

[0014] If the search result is that all searches are empty, then data is read from the backend disk;

[0015] If the search result is a complete match, then data is read from the cache disk;

[0016] If the search results are partially hit and partially empty, then the partially hit data is read from the cache disk, and the partially empty data is read from the backend disk.

[0017] The method described in the above embodiments takes into account multiple possible storage locations for data reading because the storage system utilizes a cache disk. When data can be read directly from the cache disk, it reads data directly from the cache disk instead of reading data from the back-end disk. Since the data reading rate from the cache disk is much higher than the data reading rate from the back-end disk, the above method can greatly improve the efficiency of data reading.

[0018] In one embodiment, the logical space includes a data logical space, the data processing request is the write request, and the data processing based on the key information and logical space corresponding to the data processing request, through a cache disk and a backend disk, includes:

[0019] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate the key information corresponding to the write request;

[0020] The data to be written in the write request is written to the data logical space, and the data to be written is then pushed to the back-end disk through the data block area corresponding to the data logical space.

[0021] The key information is inserted into the key information record in memory, and the key information is written to the cache disk.

[0022] The above embodiments involve the data writing process, and by allocating logical space for write requests, data is stored on the back-end disk through the cache disk. This implements a method that allows operations on the SSD disk to be performed by operating on the logical space, without the need for a cache disk mapping mechanism. Moreover, by utilizing the association between the logical space and the data block area, data can be written to the data block area in the cache disk and then copied to the back-end disk. This can significantly reduce cache disk resource overhead and also reduce write amplification caused by write requests.

[0023] In one embodiment, writing the key information to the cache disk includes:

[0024] From the identification logic space of the usage status, allocate a corresponding identification logic space for the key information;

[0025] The key information is written into the identifier logical space, and then written into the corresponding data block area in the cache disk through the identifier logical space.

[0026] The method described in the above embodiments, after storing the data corresponding to the write request on the cache disk, can also write the key information corresponding to the write request to the cache disk for permanent storage, so that the cache index can be restored when the system restarts, thereby improving the storage performance of the storage system.

[0027] In one embodiment, writing the key information to the cache disk includes:

[0028] Allocate corresponding log logic space for the key information from the log logic space in use;

[0029] The key information is written into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, all the key information in the log logical space is written into the data block area corresponding to the cache disk.

[0030] The method described in the above embodiments writes key information to the SSD disk only after a certain amount of key information has been written to the log logical space. In this way, only a log needs to be recorded after each write operation. During crash recovery, the tree structure of key information can be reconstructed through the log, avoiding the inability to read data due to key loss and improving the reliability of the storage system.

[0031] In one embodiment, the log logic space is a first log logic space, and writing the key information into the log logic space includes:

[0032] Determine if the first log logic space for the status in use has sufficient storage space;

[0033] If the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in an unused state;

[0034] If the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

[0035] The method described in the above embodiments, when it is determined that the storage space of the first log logical space is sufficient, writes key information into the first log logical space, which can reduce invalid data writes and improve the stability of the storage system.

[0036] In one embodiment, writing the key information based on the unused second log space includes:

[0037] Determine whether the second log logic space still exists;

[0038] If the second log logical space exists, the key information is written into the second log logical space;

[0039] If the second log logical space does not exist, the critical information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

[0040] The method described in the above embodiments, when it is determined that there is still an unused second log logical space, writes key information into the second log logical space, which can reduce invalid data writes and improve the stability of the storage system.

[0041] In one embodiment, the method further includes:

[0042] Generate a log write request carrying the key information and determine whether the log task should be started;

[0043] If it is determined that the log task has been started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation. Based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in the usage state has sufficient storage space is returned to be executed.

[0044] If it is determined that the log task has not been started, the log writing request is added to the temporary list, and the process returns to the step of determining whether the log task has been started.

[0045] The method described in the above embodiments, by setting log tasks and executing log processing and writing to the SSD disk through linked lists at different stages, achieves effective management of log tasks, which can improve the efficiency of log writing and the read and write performance of the storage system.

[0046] In one embodiment, writing the key information to the cache disk includes:

[0047] The key information in the memory is recorded and written to the cache disk in a tree-structured node format.

[0048] The method described in the above embodiments, by fixing the underlying disk with each node of the tree structure as a unit, can achieve orderly storage of key information, improve the efficiency of retrieving key information later, and thus improve the efficiency of data reading in the storage system.

[0049] In one embodiment, the logical space includes a data logical space, the data processing request is a write request, the write request indicates a data deletion operation or a data modification operation, and the data processing, based on the key information and logical space corresponding to the data processing request, is performed through a cache disk and a backend disk, including:

[0050] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk;

[0051] Based on the write request, allocate the corresponding data logic space and the target key information, and process the data through the cache disk and the backend disk.

[0052] The method described above can achieve efficient data deletion operations by manipulating the data logical space and setting invalid key information, and can also achieve efficient data modification operations by manipulating the data logical space and setting new key information. Compared with traditional data deletion, which requires multiple read and write operations, and traditional data modification, which requires multiple data reading and moving operations for write overwrite operations, this method can reduce write amplification and improve write operation efficiency.

[0053] In one embodiment, the write request indicates a data deletion operation, the target key information includes the invalid key information, and the process of allocating corresponding data logical space and the target key information according to the write request, and performing data processing through a cache disk, includes:

[0054] The invalid key information is inserted into the key information record in memory and written to the cache disk.

[0055] The method described above allows data to be deleted simply by inserting key information corresponding to the data to be deleted. Compared to write overwrite operations, which require multiple data reads and moves, this method can reduce write amplification and improve write operation efficiency.

[0056] In one embodiment, the write request indicates a data modification operation, the target key information includes the new key information, and the process of allocating corresponding data logical space and the target key information according to the write request, and performing data processing through a cache disk, includes:

[0057] The data to be modified by the data modification operation is written into the data logic space, and the data to be modified is then written to the back-end disk through the data block area corresponding to the data logic space.

[0058] The new key information is inserted into the key information record in memory and written to the cache disk.

[0059] The method described above allows data to be modified simply by inserting key information corresponding to the pre-modified data. Compared to write overwrite operations, which require multiple data reads and moves, this method reduces write amplification and improves write operation efficiency.

[0060] In one embodiment, the method further includes:

[0061] When the preset data space reclamation requirements are met, the cache disk is reclaimed according to the state of each data logical space in the cache disk;

[0062] When the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

[0063] The method described in the above embodiments realizes an autonomous GC space reclamation mechanism. Through this GC space reclamation mechanism, the data logic space or the identifier logic space can be defragmented and space released more efficiently, which greatly improves the reclamation efficiency and greatly reduces the need to reserve new logic space.

[0064] In one embodiment, the step of reclaiming data space from the cache disk based on the state of each data logical space in the cache disk includes:

[0065] Determine the state of each data logical space in the cache disk;

[0066] For the first data logical space in the cache disk that is in use, no data space reclamation operation is performed on the first data logical space;

[0067] For the second data logical space in the cache disk that is in a full state, data space reclamation is performed on the second data logical space according to the effective capacity of the second data logical space.

[0068] The method described in the above embodiments achieves autonomous reclamation of the data block area corresponding to the data logical space by operating on the data logical space, avoiding the preemption of user bandwidth. Moreover, the data space reclamation is all equal exchange or the exchange of old space with a small amount of new space, which does not affect the data operation process and requires a small amount of new space, which can reduce the overhead of additional resources and improve the data processing performance of the storage system.

[0069] In one embodiment, the step of reclaiming data space in the second data logical space based on its effective capacity includes:

[0070] If the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the effective data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released.

[0071] If the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation will be performed on the second data logical space.

[0072] The method described in the above embodiments achieves autonomous reclamation of the data block area corresponding to the data logical space by operating on the data logical space, avoiding the preemption of user bandwidth. Moreover, the data space reclamation is all equal exchange or the exchange of old space with a small amount of new space, which does not affect the data operation process and requires a small amount of new space, which can reduce the overhead of additional resources and improve the data processing performance of the storage system.

[0073] In one embodiment, the step of reclaiming the identifier space of the cache disk based on the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space includes:

[0074] Determine the state of each identifier logical space in the cache disk;

[0075] For the first identifier logical space in the cache disk that is in use, no identifier space reclamation operation is performed on the first identifier logical space;

[0076] For the second identifier logical space in the cache disk that is in a full state, the identifier space is reclaimed based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

[0077] The method described in the above embodiments achieves autonomous reclamation of the data block area corresponding to the identifier logical space by operating on the identifier logical space, avoiding the preemption of user bandwidth. Moreover, the data space reclamation is all equal exchange or the exchange of old space with a small amount of new space, which does not affect the data operation process and requires a small amount of new space, which can reduce the overhead of additional resources and improve the data processing performance of the storage system.

[0078] In one embodiment, the step of reclaiming data space in the second identifier logical space based on the effective capacity of the second identifier logical space and the validity of key information in the second identifier logical space includes:

[0079] If the effective capacity of the second identifier logical space is less than the second capacity threshold, then the second identifier logical space is reclaimed based on the validity of the key information in the second identifier logical space.

[0080] If the effective capacity of the second identifier logical space is not less than the second capacity threshold, then the data space reclamation operation will not be performed on the second identifier logical space.

[0081] The method described in the above embodiments achieves autonomous reclamation of the data block area corresponding to the identifier logical space by operating on the identifier logical space, and determines whether to perform space reclamation based on the effective capacity of the identifier logical space, which can improve the efficiency of space reclamation and also realizes an autonomous space reclamation mechanism.

[0082] In one embodiment, the step of reclaiming the identifier space based on the validity of key information in the second identifier logical space includes:

[0083] Determine the validity of key information in the second identifier logical space;

[0084] If all the key information in the second identifier logical space becomes invalid, then the second identifier logical space will be reclaimed.

[0085] If valid key information exists in the second identifier logical space, then a third identifier logical space is requested, and the valid key information in the second identifier logical space is moved to the third identifier logical space, and the moved second identifier logical space is released.

[0086] The method described in the above embodiments provides an autonomous CG space reclamation mechanism. By controlling the operation of logical space buckets, it can be executed during idle periods or when necessary, avoiding bandwidth contention for users. Furthermore, the CG space reclamation mechanism exchanges a majority of old logical space with an equal or small number of new logical space buckets, without affecting data operation processes and requiring very little reserved space; even with only a few buckets, the operation can be performed. Therefore, this method can significantly reduce space overhead and avoid space crowding problems, thereby improving the stability of the entire storage system.

[0087] In one embodiment, the method further includes:

[0088] Under the condition of meeting the preset reverse wear requirements, the single disk reverse wear function is activated to perform particle-level uniform wear processing on each cache disk in the storage system; and / or, the multi-disk wear function is activated to perform uniform wear processing on all cache disks in the storage system.

[0089] The method described in the above embodiments introduces an autonomous anti-wear mechanism, which is completely independent of data read and write operations and does not affect each other. It can be turned on or off autonomously, and will not occupy disk space when turned off, thereby improving the lifespan of each cache disk and the overall storage system.

[0090] In one embodiment, the single-disk reverse wear function performs particle-rate wear equalization processing on each cache disk in the storage system, including:

[0091] For each of the cache disks, obtain the number of read and write operations for each data block region in the cache disk;

[0092] The activation priority of each data block region is configured based on the number of reads and writes to each data block region in the cache disk; the fewer the number of reads and writes to a data block region, the higher the priority of the corresponding data block region.

[0093] The method described in the above embodiments has a limited number of read / write cycles for a BLOCK. When the number of read / write cycles for a BLOCK is exhausted, it will no longer be possible to write to that BLOCK, resulting in damage to the entire SSD disk containing that BLOCK. Therefore, making the granularity of each BLOCK on an SSD disk uniform can relatively improve the lifespan of the entire SSD disk.

[0094] In one embodiment, the multi-disk wear reduction function performs a rate-of-use wear reduction process on all cache disks in the storage system, including:

[0095] The remaining lifespan of each cache disk is determined based on the number of read and write operations performed on each cache disk.

[0096] The workload of each cache disk is planned based on its remaining lifespan; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

[0097] The method described in the above embodiments, since the multiple cache disks in the cache system are independent of each other, makes the lifespan exhaustion time of the multiple cache disks as different as possible. This can prevent all or multiple SSD disks in the cache system from failing due to lifespan exhaustion at the same time, thus causing the cache system to malfunction. Therefore, the above method can improve the lifespan of the cache system.

[0098] Secondly, this application also provides a data processing apparatus, comprising:

[0099] A receiving module is used to receive data processing requests; the data processing requests include read requests or write requests.

[0100] The processing module is used to process data through a cache disk based on the key information and logical space corresponding to the data processing request; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0101] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0102] Receive a data processing request; the data processing request may be a read request or a write request.

[0103] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0104] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0105] Receive a data processing request; the data processing request may be a read request or a write request.

[0106] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0107] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0108] Receive a data processing request; the data processing request may be a read request or a write request.

[0109] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0110] The aforementioned data processing method, apparatus, computer equipment, storage medium, and program product receive data processing requests and, based on the key information and / or logical space corresponding to the data processing request, perform data processing through a cache disk and a back-end disk. The logical space corresponds to a data block area in the cache disk, and the key information includes the mapping relationship between the cache disk and the back-end disk. This method introduces the concept of logical space and, through operations on this logical space, directly manipulates the cache disk without using its own data block area mapping and space reclamation mechanisms. It can autonomously complete space allocation, release, and mapping, and can also directly write to the cache disk using sequential append writes based on operations on the logical space. Furthermore, it utilizes the association between the logical space and data block areas to allocate corresponding matching data block areas, thereby significantly reducing read / write amplification and the required reserved space caused by the need for a large number of data block areas, ultimately improving the data processing performance of the storage system. In addition, the above method can introduce key information corresponding to data processing requests as an index and identifier for each piece of data, recording the correspondence between the actual written data and the data on the cache disk. This replaces the original FTL translation layer and the inherent data mapping relationship in the cache disk, reducing the maintenance of FTL, greatly reducing the additional overhead on the cache disk space, and improving the space utilization of the storage system. Attached Figure Description

[0111] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0112] Figure 1 This is one of the schematic diagrams of a data processing method for an existing SSD disk mechanism in one embodiment;

[0113] Figure 2 This is a second schematic diagram of a data processing method for an existing SSD disk mechanism in one embodiment;

[0114] Figure 3 This is the third schematic diagram of a data processing method for an existing SSD disk mechanism in one embodiment;

[0115] Figure 4 This is a fourth schematic diagram of a data processing method for an existing SSD disk mechanism in one embodiment;

[0116] Figure 5 This is a schematic diagram of the storage system of a data processing method in one embodiment;

[0117] Figure 6This is one of the flowcharts illustrating a data processing method in one embodiment;

[0118] Figure 7 This is a schematic diagram illustrating the transformation of the logical space in one embodiment;

[0119] Figure 8 This is a second flowchart illustrating a data processing method in one embodiment;

[0120] Figure 9 This is the third flowchart of a data processing method in one embodiment;

[0121] Figure 10 This is the fourth flowchart of a data processing method in one embodiment;

[0122] Figure 11 This is the fifth flowchart illustrating a data processing method in one embodiment;

[0123] Figure 12 This is a flowchart of a data processing method in one embodiment, number six.

[0124] Figure 13 This is the seventh flowchart of a data processing method in one embodiment;

[0125] Figure 14 This is the eighth flowchart of a data processing method in one embodiment;

[0126] Figure 15 This is the ninth flowchart of a data processing method in one embodiment;

[0127] Figure 16 This is a flowchart of a data processing method in one embodiment, number ten.

[0128] Figure 17 This is eleventh of a flowchart illustrating a data processing method in one embodiment;

[0129] Figure 18 This is a flowchart of a data processing method in one embodiment, number 12.

[0130] Figure 19 This is a flowchart of a data processing method in one embodiment, number thirteen.

[0131] Figure 20 This is a flowchart of a data processing method in one embodiment, number fourteen.

[0132] Figure 21 This is one of the flowcharts illustrating the logic space reclamation process in one embodiment;

[0133] Figure 22 This is a flowchart of a data processing method in one embodiment, number fifteen.

[0134] Figure 23 This is a flowchart of a data processing method in one embodiment, number sixteen.

[0135] Figure 24 This is the second schematic diagram of the logic space reclamation process in one embodiment;

[0136] Figure 25 This is a flowchart of a data processing method in one embodiment, number seventeen.

[0137] Figure 26 This is one of the schematic diagrams illustrating the wear effect of the cache disk in one embodiment;

[0138] Figure 27 This is the eighteenth flowchart of a data processing method in one embodiment;

[0139] Figure 28 This is a second schematic diagram illustrating the wear effect of the cache disk in one embodiment;

[0140] Figure 29 This is a structural block diagram of a data processing device in one embodiment;

[0141] Figure 30 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0142] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0143] Most current storage systems use solid state drives (SSDs) for caching as a means of acceleration. However, SSD usage largely relies on the SSD's own space scheduling, meaning data is written directly with the given address, and read out with the given address and size. While current storage products have achieved improved read / write throughput through SSDs, performance and hardware utilization efficiency could still be improved. For example, when an SSD is used directly, there are internal Flash Translation Layer (FTL) address mapping, garbage collection (GC) mechanisms, and over-provisioning (OP) space. Specifically: FTL address mapping incurs additional space overhead due to the need to store mapping information; simultaneously, due to the SSD's block reclamation characteristics, during space reclamation, garbage collection (GC) must move valid data to other blocks to ensure no data loss; data movement requires a certain amount of free OP space, and reserving OP space can lead to the compression of available disk space; furthermore, data movement caused by GC will preempt disk read / write bandwidth, affecting actual read / write speeds; additionally, data movement caused by GC will result in additional read / write operations, accelerating the lifespan consumption of SSD storage particles. For example, see... Figures 1-4 The data processing method of the existing SSD disk mechanism is shown, wherein, Figure 1 Originally, there were four data segments on the disk, which were then distributed across multiple blocks after FTL mapping; to delete data segment 1, see [link to relevant documentation]. Figures 2-4 The SSD requires GC space reclamation of all blocks containing data segment 1, which uses a large amount of OP block space. GC space reclamation of all blocks containing data segment 1 is performed, and the remaining data is moved to OP block space. Reclaiming data segment 1 affects all blocks, replacing OP block space, and completing the deletion. Based on the above operations, it was found that one deletion operation results in reading 4 blocks and writing 3 blocks, requiring 3 blocks as OP space. It is evident that there is a problem of write amplification and a large amount of additional space overhead.

[0144] As mentioned above, current storage products utilize the advantages of SSDs—high concurrency and high bandwidth—to improve read and write throughput. However, this also results in performance degradation due to space mapping and garbage collection (GC) issues caused by the high concurrency and bandwidth of SSDs, leading to write amplification and additional read / write cycles. This, in turn, accelerates the lifespan of the SSDs. In short, existing SSD caching mechanisms suffer from a high cost-effectiveness problem.

[0145] In view of this, embodiments of this application propose a data processing method, apparatus, computer equipment, storage medium, and program product that can directly operate on SSD disks without using the SSD disk's own mapping and GC, and can autonomously complete the allocation, release, and mapping of space, thereby significantly reducing write amplification and SSD disk resource overhead, and thus improving the lifespan of SSD disks.

[0146] It should be noted that the beneficial effects or technical problems solved by the embodiments of this application are not limited to this one, but may also be other implicit or related problems. For details, please refer to the description of the embodiments below.

[0147] The data processing method provided in this application embodiment can be applied to, for example... Figure 5 The storage system shown includes a memory region, multiple cache SSDs, and multiple back-end HDDs. The memory region stores key information corresponding to each batch of data processing requests. Each cache SSD contains various types of logical space buckets, such as data logical space buckets, identifier logical space buckets, and log logical spaces. Each cache SSD also contains multiple data block regions (BLOCKs), with each BLOCK corresponding to a logical space bucket. Each cache SSD is a cache disk based on the ZNS mechanism. This storage system can be a storage system on a computer device or a storage system corresponding to a distributed storage server cluster. Figure 5 The number of buckets and blocks shown is for illustrative purposes only, and there is no limit to the number of buckets and blocks. Figure 5 Only one cache disk SSD is shown here for illustrative purposes only; there is no limit to the number of cache disk SSDs.

[0148] In one exemplary embodiment, such as Figure 6 As shown, a data processing method is provided, which can be applied to... Figure 5 Taking the storage system in the example of [the example], the explanation includes:

[0149] S201, Receive data processing request.

[0150] Data processing requests include read requests or write requests. Write requests are used to indicate data deletion or data modification operations.

[0151] In this embodiment, the storage system can write data to the hard disk drive (HDD) via memory and cache SSD, delete data from the HDD via memory and cache SSD, modify data from the HDD via memory and cache SSD, and read data from the HDD via memory and cache SSD. Specifically, the controller in the SSD can receive data processing requests from the upper layer or from external sources, and then perform corresponding read, write, delete, or modify operations on the data according to the type of data processing request.

[0152] S202, based on the key information and logical space corresponding to the data processing request, performs data processing through the cache disk and backend disk.

[0153] In this application, logical space can be labeled as a bucket, and data block area can be labeled as a BLOCK. Logical space corresponds to data block area in cache disk. Key information includes the mapping relationship between cache disk and backend disk. Logical space can be bucket space, corresponding to the concept of zone partition in ZNS, representing space allocated on the SSD disk. In this embodiment, logical space can be divided into three types: data logical space, identifier logical space, and log logical space. The size of the bucket space can correspond to or be adjusted according to the size of the data block area BLOCK in the SSD disk, maintaining consistency with the BLOCK size. Optionally, one bucket space can correspond to one BLOCK space of the same size. Optionally, one bucket space can also correspond to multiple BLOCK spaces, and the sizes of the multiple BLOCK spaces correspond to the size of the aforementioned one bucket space. The state of the bucket space can be divided into three states: unused (labeled as unused), in use (labeled as open), and full (labeled as inuse). The transition paths for these three states are described in [link to relevant documentation]. Figure 7 As shown, when a bucket space is enabled, the `zoneAppend` command can allocate bucket space on the SSD, replacing the existing SSD driver. The bucket space then enters a state of use, and its space can only be used sequentially. Once the bucket space is used up, it enters a write-full state, and thereafter remains in a read-only state until it is evicted or reclaimed by a self-managed garbage collector (GC). When a bucket space is reclaimed, the `zoneTrim` command instructs the SSD to erase the block corresponding to that bucket space, making it writable, thus reclaiming the block space on the SSD.

[0154] Optionally, critical information is represented using a key. Each piece of data written to a bucket is marked with a key to provide an index for lookup and retrieval. The critical information key contains the HDD number and address where the data should be written, as well as the length information; it also includes the data's number, location, and length information on the SSD; the critical information also includes the iteration number or lock identifier of the corresponding cached data bucket, which is used to determine whether the cached data is valid.

[0155] Optionally, for ease of searching and maintenance, the key information stored in memory can be integrated and managed using tree structures such as B+ trees, multi-way trees, and red-black trees. Different structures have their own advantages and disadvantages, and the choice can be made according to actual needs. There is no limitation on the storage structure of the key. The tree structure is collectively referred to as a tree, and the key information can be written to the key bucket for hard disk storage, node by node of the tree.

[0156] Optionally, operations on key information can be categorized as follows: lookup (e.g., during read operations in data operations, the key needs to be looked up to determine the storage address of the data to be read), traversal (e.g., during cache flushing operations, the mapping relationship from SSD to HDD needs to be determined based on the key to facilitate transferring data from SSD to HDD), insertion (e.g., during write operations in data operations, the key corresponding to the data being written is directly inserted), and compaction (e.g., during discretionary GC, the key is updated accordingly). In this embodiment, to ensure the efficiency of SSD usage, key operations only involve insertion, without modification or deletion, thus decoupling the strong binding between data operations and garbage collection (GC).

[0157] Optionally, all key information will be stored on an SSD so that the cache index can be restored when the system restarts.

[0158] In this embodiment, when the storage system receives a data processing request, it can first determine the type of the data processing request. In the first scenario, if the data processing request is a read request, the address index of the HDD is extracted from the read request, and the key information corresponding to the data processing request is searched in memory based on the address index. Since this key information includes the mapping relationship between the cache SSD and the back-end HDD, it also includes the number, location, and length information of the cache SSD; and the number, address, and length information of the back-end HDD. Therefore, the data to be read by the data processing request can be read from the cache SSD or HDD based on this key information. In the second scenario, if the data processing request is a write request, the data to be written is written to the data logical space bucket in the SSD, and a key information corresponding to the data is generated. This key records the number and address of the SSD where the data is located, as well as the length information, and also records the number, address, and length information of the HDD to which the data should be written. The key information corresponding to the data is then saved, specifically in memory, for easy reference when reading the data next time. All key information in memory can be stored in a tree-structured data structure for easy key retrieval. Optionally, the key information in memory can also be permanently stored in the identifier logical space bucket of the SSD, so that the key can be restored during system restart for easy indexing. The third scenario is that if the data processing request is a write request and corresponds to a data deletion operation, an invalid key information corresponding to that data deletion operation is generated. This invalid key information does not point to any data block area BLOCK on the SSD; that is, the invalid key information only includes the HDD disk's number, address, and length information, and does not contain SSD disk information. The fourth scenario involves a write request. If the write request corresponds to a data modification operation, the data to be modified is written to the data logical space bucket on the SSD. A key is generated corresponding to this modified data, recording the SSD's ID and address, length, and the HDD ID, address, and length information where the modified data should be written. This key is then saved in memory for easy access when retrieving the data later. It's crucial that the storage system writes data to the SSD sequentially, not randomly. This can be achieved using methods such as... Figure 7 The space conversion process shown is written via bucket.

[0159] The data processing method described in the above embodiments receives a data processing request and processes the data through a cache disk and a back-end disk based on the key information and logical space corresponding to the data processing request. The logical space corresponds to a data block area in the cache disk, and the key information includes the mapping relationship between the cache disk and the back-end disk. This method introduces the concept of logical space and directly operates on the cache disk without using its own data block area mapping and space reclamation mechanisms by manipulating the logical space. It can autonomously complete space allocation, release, and mapping, and can also directly write to the cache disk using sequential append writes based on operations on the logical space. Furthermore, it allocates corresponding matching data block areas by utilizing the association between the logical space and the data block area, thereby significantly reducing read / write amplification and the required reserved space caused by the need for a large number of data block areas, and thus improving the data processing performance of the storage system. In addition, the above method can introduce key information corresponding to data processing requests as an index and identifier for each piece of data, recording the correspondence between the actual written data and the data on the cache disk. This replaces the original FTL translation layer and the inherent data mapping relationship in the cache disk, reducing the maintenance of FTL, greatly reducing the additional overhead on the cache disk space, and improving the space utilization of the storage system.

[0160] In one embodiment, when the data processing request is a read request, a data reading process is provided for that read request, which is one implementation of the above-mentioned S202 "processing data through a cache disk based on the key information and logical space corresponding to the data processing request," such as... Figure 8 As shown, the method includes:

[0161] S301: Based on the address index carried in the read request, retrieve the key information from the key information record in memory and obtain the retrieval result.

[0162] The address index refers to the address index of the HDD. Key information records include key information corresponding to multiple data entries, structured in a tree structure. Search results include three types: all results found, all results found but not found, and some results found but not found.

[0163] In this embodiment, when the storage system receives a read request, it can extract the address index of the HDD from the read request and search for the key information corresponding to the read request from the key information record in memory based on the address index. Since the data to be read by the read request may be on the SSD, on the HDD, or partially on the SSD and partially on the HDD, three search results can be obtained for these three situations: when the data to be read by the read request is on the SSD, the search result includes all search hits; when the data to be read by the read request is on the HDD, the search result includes all search results being empty; and when some of the data to be read by the read request is on the SSD and some is on the HDD, the search result includes some search hits and some search results being empty.

[0164] S302, based on the search results, reads data through the cache disk and the backend disk.

[0165] In this embodiment of the application, when the storage system obtains the retrieval results, since the retrieval results can indicate where to read the data, and since the key information corresponding to the read request includes the mapping relationship between the cache disk and the back-end disk, it also includes the number, location, and length information of the cache disk SSD; and the number, address, and length information of the back-end disk HDD. Therefore, the data to be read by the data processing request can be read from the cache disk SSD or HDD based on this key information and in combination with the retrieval results.

[0166] The above embodiments involve a data reading process, and the location of the data to be read is determined by retrieving key information corresponding to the read request. This method differs from the existing storage products that determine the data reading location after mapping based on the FTL translation layer. This method is simple, does not require additional space overhead, and can greatly improve the efficiency of data reading.

[0167] In one embodiment, a data reading process is provided, namely, an implementation of the above-mentioned S302 "reading data through the cache disk based on the search results", such as... Figure 9 As shown, the method includes:

[0168] S401, Determine the search results. If the search results are all empty, proceed to step S402; if the search results are all hits, proceed to step S403; if the search results are some hits and some empty, proceed to step S404.

[0169] S402 reads data from the back-end disk.

[0170] S403 reads data from the cache disk.

[0171] S404 reads the partially retrieved data from the cache disk and the partially retrieved data from the backend disk.

[0172] In this embodiment, when the storage system obtains the search results, it can determine the search results. If the search results are all empty, it means that the data to be read is not on the SSD but is on the HDD. Therefore, the data to be read can be read directly from the HDD and then returned to the user terminal, while also being written to the cache disk. If the search results are all hits, it means that the data to be read is on the SSD. Therefore, the data to be read can be read directly from the cache SSD and then returned to the user terminal. If the search results are partially hits and partially empty, it means that some of the data to be read is on the SSD and some is on the HDD. Therefore, some data can be read from the cache SSD and some data can be read from the HDD. Then, the two parts of the read data are concatenated and returned to the user terminal.

[0173] The method described in the above embodiments takes into account multiple possible storage locations for data reading because the storage system utilizes a cache disk. When data can be read directly from the cache disk, it reads data directly from the cache disk instead of reading data from the back-end disk. Since the data reading rate from the cache disk is much higher than the data reading rate from the back-end disk, the above method can greatly improve the efficiency of data reading.

[0174] In one embodiment, when the data processing request is a write request, a data writing process is provided for that write request, which is an implementation of the above-mentioned S202 "data processing is performed through the cache disk and backend disk according to the key information and logical space corresponding to the data processing request". Figure 10 As shown, the method includes:

[0175] S501: Allocate a corresponding data logic space for the write request from the data logic space in the usage state, and generate the key information corresponding to the write request.

[0176] The logical space can include a data logical space, an identifier logical space, and a log logical space. The state of the data logical space can be any of the following: in use, unused, or full.

[0177] In this embodiment, when the storage system receives a write request, it can check the status of all data logical spaces and allocate a corresponding data logical space from multiple data logical spaces that are in use for the data to be written. Here, the data logical space can be a data logical space bucket. A corresponding data block area (BLOCK) or multiple data block areas (BLOCKs) are also allocated in the cache disk. For details regarding the sequential writing of data in the data logical space, please refer to the aforementioned... Figure 7 The space conversion process continues. Then, the storage system generates key information corresponding to the write request based on the disk number and address of the cache disk where the data to be written is located, the length of the data to be written, and the disk number and address of the HDD disk to which the data should be written, as well as the length of the data to be written. This key information can be used to read the data later.

[0178] S502 writes the data to be written in the write request to the data logical space, and then writes the data to the backend disk through the data block area corresponding to the data logical space.

[0179] In this embodiment, when the storage system requests resources for the data logical space (i.e., requests the data logical space and the corresponding data block area), it can further sequentially write the pre-written data into the data logical space, and then the data logical space sequentially writes the data into the corresponding data block area. This achieves the operation of the data logical space to write data into the data block area of ​​the cache disk. Then, the data in the data block area can be mapped to the backend disk for storage.

[0180] S503, insert the key information into the key information record in memory, and write the key information to the cache disk.

[0181] Among them, the key information record records multiple key information keys corresponding to existing data in a tree structure.

[0182] In this embodiment, after generating the key information corresponding to the pre-written data based on the aforementioned steps, the key information can be inserted into the key information record in memory. For example, a key can be inserted as a leaf node in a tree structure's key information record. This updates the key information record in memory when writing data, ensuring that the key information record always matches the existing data, allowing for direct retrieval of the key from memory to read data later. Alternatively, the key information can be written to a cache disk, facilitating retrieval of the key from the cache disk for later data reading. This maintains sequential writing during persistent disk operations on the key information. Furthermore, when maintaining the tree structure's keys, new keys can easily replace old keys, ensuring that only the latest key is valid during queries.

[0183] The above embodiments involve the data writing process, and by allocating logical space for write requests, data is stored on the back-end disk through the cache disk. This implements a method that allows operations on the SSD disk to be performed by operating on the logical space, without the need for a cache disk mapping mechanism. Moreover, by utilizing the association between the logical space and the data block area, data can be written to the data block area in the cache disk and then copied to the back-end disk. This can significantly reduce cache disk resource overhead and also reduce write amplification caused by write requests.

[0184] In one embodiment, a method for writing critical information to a cache disk is provided, such as... Figure 11 As shown, the method includes:

[0185] S601, allocate corresponding identifier logic space for key information from the identifier logic space of the usage status.

[0186] The identifier logical space can represent a key bucket. The state of the identifier logical space can be any of the following: in use, unused, or full.

[0187] In this embodiment, after the storage system writes the pre-written data to the data logical space, it can view the status of all identifier logical spaces and allocate a corresponding identifier logical space for the key information corresponding to the write request from multiple identifier logical spaces in use. A corresponding data block area (BLOCK) is also allocated in the cache disk corresponding to the identifier logical space. For details on writing the key information to the identifier logical space, please refer to the aforementioned... Figure 7 The spatial transformation process continues.

[0188] S602 writes the key information into the identifier logical space, and then writes the key information into the corresponding data block area in the cache disk through the identifier logical space.

[0189] In this embodiment, when the storage system requests an identifier logical space, that is, requests both the identifier logical space and the corresponding data block area, key information can be further written to the identifier logical space, and then the identifier logical space writes the key information to the corresponding data block area. This achieves the goal of writing key information to the data block area of ​​the cache disk through operations on the identifier logical space. Sequential writing can also be maintained when performing persistent disk write operations on the key information y.

[0190] The method described in the above embodiments, after storing the data corresponding to the write request on the cache disk, can also write the key information corresponding to the write request to the cache disk for permanent storage, so that the cache index can be restored when the system restarts, thereby improving the storage performance of the storage system.

[0191] In one embodiment, another method for writing critical information to a cache disk is provided, such as... Figure 12 As shown, the method includes:

[0192] S701 allocates corresponding log logical space for key information from the log logical space in use.

[0193] The state of the log logic space can be any of the following: in use, unused, or full.

[0194] In this embodiment, if the key information is written to the SSD every time data is written to the cache disk, it will cause data write amplification and affect data operation efficiency. If the key is written late, it will be lost in the event of a crash. Therefore, this embodiment introduces a logging mechanism. After data is written, the corresponding key is logged in sequence. So when writing data to the cache disk and generating the key information key corresponding to the written data, a corresponding logical space, i.e., a log logical space, can be allocated for the key information key first. Specifically, the status of the log logical space can be checked first, and the corresponding log logical space for the key information corresponding to the write request can be allocated from multiple log logical spaces in the use state. It should be noted that the storage system in this embodiment has a log logical space specifically planned for logs, and it can be used cyclically. When the log logical space bucket in the use state is full, the next log logical space bucket in the unused state will be automatically started.

[0195] S702 writes key information into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, writes all the key information in the log logical space into the data block area corresponding to the cache disk.

[0196] The preset quantity threshold can be determined according to actual storage needs and is used to measure the frequency of writing key information.

[0197] In this embodiment of the application, after writing data, the key information corresponding to the written data can be written sequentially to the log logical space according to the order in which the written data was stored. The number of data contained in the log logical space is detected. When the number reaches a preset threshold, all the key information in the log logical space can be written sequentially to the data block area corresponding to the cache disk, thereby fixing the key information to the cache disk.

[0198] The method described in the above embodiments writes key information to the SSD disk only after a certain amount of key information has been written to the log logical space. In this way, only a log needs to be recorded after each write operation. During crash recovery, the tree structure of key information can be reconstructed through the log, avoiding the inability to read data due to key loss and improving the reliability of the storage system.

[0199] In one embodiment, a method for writing logs is provided, namely, "writing key information into the log logical space" in S702 above, such as... Figure 13 As shown, it includes:

[0200] S801, determine whether the storage space of the first log logical space in use is sufficient. If the storage space of the first log logical space is insufficient, proceed to step S802; if the storage space of the first log logical space is sufficient, proceed to step S803.

[0201] S802, write critical information into the second log logical space that is not in use.

[0202] S803 writes key information into the first log logical space.

[0203] In this embodiment of the application, when allocating a corresponding first log logical space for key information based on the aforementioned steps, the state of the first log logical space is in use. Then, it can be determined whether the space in the first log logical space is sufficient. If it is sufficient, it means that the key information can be written into the first log logical space, so the key information is written into the first log logical space. If it is insufficient, it means that the first log logical space cannot store the key information, and a new log logical space needs to be selected to write the key information. Therefore, a second log logical space in an unused state is selected to write the key information. It should be noted that the method for determining whether the first log logical space has sufficient storage space can be determined by checking the capacity of the remaining storage space in the first log logical space. For example, if the capacity of the remaining storage space is less than a preset capacity threshold, then it is determined whether the first log logical space has insufficient storage space; if the capacity of the remaining storage space is less than or equal to the preset capacity threshold, then it is determined whether the first log logical space has sufficient storage space. Alternatively, the method for determining whether the first log logical space has sufficient storage space can be determined by checking whether the capacity of the remaining storage space in the first log logical space is less than the amount of data of the key information corresponding to the write request. If it is less, then it is determined whether the first log logical space has insufficient storage space; if it is not less, then it is determined whether the first log logical space has sufficient storage space.

[0204] The method described in the above embodiments, when it is determined that the storage space of the first log logical space is sufficient, writes key information into the first log logical space, which can reduce invalid data writes and improve the stability of the storage system.

[0205] In one embodiment, an implementation of the above-mentioned S802 "writing key information according to the unused state of the second log logical space" is provided, such as... Figure 14 As shown, this implementation method includes:

[0206] S901, determine whether a second log logical space still exists. If a second log logical space exists, proceed to step S902; if a second log logical space does not exist, proceed to step S903.

[0207] S902, writes key information into the second log logical space.

[0208] S903 triggers the removal of critical information from the first log logical space to the cache disk and reclaims the first log logical space.

[0209] In this embodiment, when it is determined that the storage space of the first log logical space is insufficient based on the aforementioned steps, it is possible to check whether there is an unused second log logical space. If it exists, it means that the key information can be written to the second log logical space, so the key information is written to the second log logical space. If it does not exist, it means that there is no suitable log logical space to store the key information, so the key information in the first log logical space can be triggered to be downloaded to the cache disk. When all the key information is successfully downloaded to the cache disk, the first log logical space is reclaimed, that is, the space in the first log logical space is released. After that, the process can return to the above step S801 "determine whether the storage space of the first log logical space in use is sufficient".

[0210] The method described in the above embodiments, when it is determined that there is still an unused second log logical space, writes key information into the second log logical space, which can reduce invalid data writes and improve the stability of the storage system.

[0211] In one embodiment, the logging mechanism introduced in this application includes a "log task" for performing log processing and writing to the SSD disk. Figure 13 The method described in the embodiments, such as Figure 15 As shown, it also includes:

[0212] S1001, Generate a log write request carrying key information and determine whether the log task has started; if the log task has started, proceed to step S1002; if the log task has not started, proceed to step S1003.

[0213] S1002, add the log write request to the processing linked list, extract the log write requests of the preset capacity from the processing linked list and aggregate them, and return to the step of determining whether the first log logical space in use has enough storage space based on the key information corresponding to the aggregation result.

[0214] S1003, add the log write request to the temporary list and return to execute the step of determining whether the log task should be started.

[0215] In this embodiment, the log mechanism includes a "log task" for performing log processing and writing to the SSD; the log mechanism also includes a "temporary list" for temporarily storing key information to be written to the log; the log mechanism also includes a "processing list" for storing key information being processed; and the log mechanism also includes an "aggregation list" for storing key information to be aggregated. The storage system can utilize various log tasks or linked lists within a queued log mechanism. When writing critical information to the log using this mechanism, a log write request carrying the critical information can be generated and added to a temporary linked list. First, it can be determined whether the log task has started. If so, requests are retrieved from the temporary linked list and added to the processing linked list. Then, a certain number of log write requests are retrieved from the processing linked list, their size is calculated, and it is ensured that it does not exceed a preset capacity threshold, such as 32KB. These retrieved log write requests are then added to the aggregation linked list to begin aggregation. After aggregation, the aforementioned step S801 can be executed: determining whether the space in the log logical space of the current log in use is sufficient. If the space in the log logical space bucket of the current log in use is sufficient, or if the space in the bucket of the current log in use is insufficient but other log buckets are available, the aggregation result is asynchronously processed and written to the current log in use bucket. If no other log logical space buckets are available, log reclamation is performed, and the process is repeated after reclamation. As mentioned above, if it is determined that the log task has not started, it is necessary to wait for the log task to start. Therefore, the log writing request is added to the temporary linked list for waiting, and the log task is continuously checked for startup until it starts, and then the process after the log task starts is executed.

[0216] The method described in the above embodiments, by setting log tasks and executing log processing and writing to the SSD disk through linked lists at different stages, achieves effective management of log tasks, which can improve the efficiency of log writing and the read and write performance of the storage system.

[0217] In one embodiment, then Figure 10 After "inserting key information into key information records in memory" in S503 of the embodiment, all key information records in memory can be written to the cache disk. The specific method may include: writing the key information records in memory to the cache disk in units of tree-structured nodes.

[0218] In this embodiment, since the key information record includes key information from multiple data entries, and to facilitate key retrieval and maintenance, tree structures such as B+ trees, multi-way trees, and red-black trees can be used to integrate and manage the keys. Different structures have their own advantages and disadvantages, and can be selected according to actual needs. Therefore, when recording this key information to the cache disk, it can be written to the identifier logical space key bucket for disk storage, using tree nodes as units. The specific method for disk storage can be found in the aforementioned... Figures 12-15 The method described in the embodiments. The above method, by fixing each node of the tree structure as a unit on the disk, can achieve orderly storage of key information, improve the efficiency of later retrieval of key information, and thus improve the efficiency of data reading in the storage system.

[0219] In summary Figures 12-15 The method described in the embodiments provides a log writing method, such as... Figure 16 As shown, the method includes:

[0220] S1101, Determine whether the log task has started. If the log task has started, proceed to steps S1102-S1108; if the log task has not started, proceed to step S1109.

[0221] S1102, add the log writing request to the processing linked list.

[0222] S1103, extract log write requests of a preset capacity from the linked list in the processing and aggregate them.

[0223] S1104. Determine whether the first log logical space in the usage state has sufficient storage space. If it is insufficient, proceed to steps S1105-S1107; if it is sufficient, proceed to step S1108.

[0224] S1105, determine whether a second log logical space still exists. If a second log logical space exists, proceed to step S1106; if a second log logical space does not exist, proceed to step S1107.

[0225] S1106, write the key information into the second log logical space.

[0226] S1107 triggers the removal of critical information from the first log logical space to the cache disk, reclaims the first log logical space, and returns to the execution of step S1104.

[0227] S1108, write the key information into the first log logical space.

[0228] S1109, add the log write request to the temporary list, and return to execute step S1101.

[0229] Each of the above steps has been explained in the preceding content; please refer to the preceding content for detailed explanations, which will not be repeated here. It should be noted that when the key information corresponding to the log logical space is successfully written to the SSD disk, the log logical space and the data block area corresponding to the log logical space can be reclaimed. When all logs in a full log logical space are reclaimable, the log logical space and the data block area corresponding to the log logical space can be reclaimed.

[0230] In one embodiment, when the data processing request is a write request, this write request can correspond to either a data deletion operation or a data modification operation. Based on this, a data processing method is provided, namely... Figure 6 In the embodiment, S202 "based on the key information and logical space corresponding to the data processing request, data processing is performed through the cache disk and the backend disk," as shown below. Figure 17 As shown, it includes:

[0231] S1201: Allocate a corresponding data logic space for the write request from the data logic space in the usage state, and generate the target key information corresponding to the write request.

[0232] The target key information includes invalid key information or new key information. Invalid key information includes information about the backend disk, while new key information includes information about both the backend disk and the new cache disk. Invalid key information is an index or identifier that points only to the backend disk and not to any cache disk SSD range. The backend disk information includes the disk number and address of the HDD where the data to be deleted resides, the length of the written data, etc. Data deletion operations only correspond to the deletion and replacement of the key information of the data to be deleted, i.e., generating an invalid key information key and then overwriting the original key information corresponding to the deleted data with the invalid key information. The new cache disk information includes the disk number and address of the SSD where the data to be modified resides, the length of the written data, etc. The backend disk information includes the disk number and address of the HDD where the data to be deleted resides, the length of the written data, etc. The new key information also includes the mapping relationship between the new cache disk and the backend disk. Data modification operations correspond to the write operation of the data to be modified and the write operation of the key information key corresponding to the data to be modified.

[0233] In this embodiment, when the storage system receives a write request indicating a data deletion operation, it generates invalid key information corresponding to the data deletion operation based on the disk number and address of the HDD where the data to be deleted is located, the length information of the data to be written, etc. This generates target key information including the invalid key information key for the data deletion operation. If the write request indicates a data modification operation, the system can view the status of all data logical spaces and allocate corresponding data logical spaces from multiple data logical spaces in use, as well as the data to be modified for the data modification operation. Here, the data logical space can be a data logical space bucket. A data block area (BLOCK) or multiple data block areas (BLOCKs) corresponding to this data logical space are also allocated in the cache disk. Specifically, after writing the data to be modified in the data logical space, please refer to the aforementioned... Figure 7 The space transformation process continues. Then, the storage system generates target key information corresponding to the write request based on the disk number and address of the new cache disk corresponding to the pre-modified data, the length information of the written data, and the disk number and address of the HDD disk where the written data is located, as well as the length information of the written data. This key information can be used to read the data later.

[0234] S1202 allocates the corresponding data logic space and target key information according to the write request, and processes the data through the cache disk and backend disk.

[0235] In this embodiment, when the storage system receives a write request corresponding to a data deletion operation, it generates target key information corresponding to the data deletion operation, including an invalid key information key. This invalid key information key does not point to any data block area (BLOCK) on the SSD; that is, the invalid key information key only includes the HDD disk's number, address, and length information, excluding SSD disk information. If the write request corresponds to a data modification operation, the system writes the data to be modified to the data logical space bucket on the SSD disk and generates target key information corresponding to the data to be modified, including a new key information key. It records the SSD disk's number, address, and length information, as well as the HDD number, address, and length information that the data to be modified should be written to. Then, it saves the new key information key corresponding to the data to be modified, specifically in memory for easy access when reading the data again. It is important that when the storage system writes data to the SSD disk, it uses a sequential append method, not random writing. This can be achieved using methods such as... Figure 7 The space conversion process shown is written via bucket.

[0236] In one embodiment, when the data processing request is a write request, this write request can correspond to a data deletion operation. This section describes the scenario for a data deletion operation, namely... Figure 17 S1202 in the embodiment, "allocating the corresponding data logic space and target key information according to the write request, and performing data processing through the cache disk and the backend disk", includes: inserting invalid key information into the key information record in memory, and writing it to the cache disk.

[0237] In this embodiment, when the storage system receives a write request indicating a data deletion operation, it can choose not to perform either the write or deletion operation. Instead, it can directly generate an invalid key and locate the original key corresponding to the data to be deleted. Then, it replaces the original key with the invalid key and inserts it into the key information record. Since the invalid key does not point to any interval on the cache disk, subsequent readings of the data to be deleted cannot retrieve it from the cache disk, thus achieving data deletion. For example, inserting a key as a leaf node in a tree structure's key information record updates the key information record in memory when writing data, ensuring the key information record always matches the existing data. This allows for direct retrieval of the key from memory when reading data later. For instance, to delete a key representing HDD interval A, a key that does not point to any SSD interval is inserted. Using this scheme, the write amplification factor caused by data overwrite and deletion operations is almost close to 1, significantly reducing write amplification.

[0238] The method described above allows data to be deleted simply by inserting key information corresponding to the data to be deleted. Compared to write overwrite operations, which require multiple data reads and moves, this method can reduce write amplification and improve write operation efficiency.

[0239] In one embodiment, when the data processing request is a write request, this write request can also correspond to a data modification operation. This section describes the scenario of a data modification operation, namely... Figure 17 In the embodiment, S1202 "allocates the corresponding data logic space and target key information according to the write request, and performs data processing through the cache disk and backend disk," as shown below. Figure 18 As shown, it includes:

[0240] S1301, writes the data to be modified by the data modification operation into the data logic space, and then uses the data logic space to push the data to the back-end disk through the data block area corresponding to the data logic space.

[0241] In this embodiment, when the storage system receives a write request indicating a data modification operation, it can view the status of all data logical spaces and allocate a corresponding data logical space from multiple data logical spaces in use for the data to be modified. This data logical space can be a data logical space bucket. A corresponding data block area (BLOCK) or multiple data block areas (BLOCKs) are also allocated in the cache disk. After writing the data to be modified into the data logical space, please refer to the aforementioned... Figure 7 The space transformation process continues. Then, the storage system generates key information corresponding to the write request based on the disk number and address of the new cache disk corresponding to the pre-modified data, the length of the written data, and the disk number and address and length of the HDD disk where the written data resides. This key information is used to retrieve the data later. When the storage system requests data logical space resources (i.e., requests the data logical space and its corresponding data block area), it can further write the pre-modified data to the data logical space, which then writes the data to the corresponding data block area. This process of writing the pre-modified data to the cache disk's data block area is achieved. Finally, the pre-modified data in the data block area can be overwritten onto the backend disk, thus modifying the data on the backend disk.

[0242] S1302 inserts new key information into the key information record in memory and writes it to the cache disk.

[0243] The new key information includes information about the backend disk and the new cache disk. Specifically, the new key information is an index or identifier pointing to both the backend disk and a new range within the cache SSD. The new cache disk information includes the drive number and address of the SSD where the data to be modified will be written, as well as the length of the data to be written. The backend disk information includes the drive number and address of the HDD where the data to be modified resides, and the length of the data to be written.

[0244] In this embodiment, after generating new key information corresponding to the pre-modified data based on the aforementioned steps, the new key information key can be inserted into the key information record in memory to modify the key information key corresponding to the pre-modified data. For example, a key can be inserted as a leaf node in a tree structure into the key information record, updating the key information record in memory when writing data, ensuring that the key information record always matches the existing data, so that the key can be directly retrieved from memory to read data later. On the other hand, the new key information key can be written to a cache disk, facilitating retrieval of the key from the cache disk for data reading later. For example, if a key representing HDD range B needs to be modified, the new data is written to the currently used bucket, and then a key pointing to the new SSD range is inserted.

[0245] The method described above allows data to be modified simply by inserting key information corresponding to the pre-modified data. Compared to write overwrite operations, which require multiple data reads and moves, this method reduces write amplification and improves write operation efficiency.

[0246] In one embodiment, since the data or information in the logical space in all the above embodiments is written sequentially—for example, the data written by the data processing request is written sequentially to the data logical space, or the key information corresponding to the data processing request is written sequentially to the identifier logical space—when an overwrite or deletion occurs, the invalidated data will still remain in the data logical space bucket that is at the beginning of the sequence. If it is not cleaned up and reclaimed, it will occupy SSD disk space. Therefore, this application also provides an autonomous GC mechanism, i.e., a space reclamation mechanism, such as... Figure 19 As shown, the method includes:

[0247] S1401, when the preset data space reclamation requirements are met, performs data space reclamation on the cache disk according to the status of each data logical space in the cache disk.

[0248] Among them, the preset data space reclamation requirement is essentially the trigger condition for data space reclamation. The preset data space reclamation requirement can be determined according to the performance requirements of the storage system. For example, the preset data space reclamation requirement can be set as the requirement to reclaim space at preset intervals, or the preset data space reclamation requirement can be triggered when the data logical space is full, or the preset data space reclamation requirement can be triggered when the preset data space is defragmented, or the preset data space reclamation requirement can be the requirement to optimize cache disk space.

[0249] In this embodiment of the application, the storage system can detect whether the cache disk meets the preset data space reclamation requirements. When it is determined that the preset data space reclamation requirements are met, it means that the space of the cache disk needs to be optimized, or that the space of the cache disk is at risk of being squeezed. Therefore, the data logical space in the cache disk that is full can be filtered out and the filtered data logical space can be reclaimed by CG.

[0250] S1402, when the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the status of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

[0251] Among them, the preset identifier space reclamation requirement is essentially the trigger condition for identifier space reclamation. The preset identifier space reclamation requirement can be determined according to the performance requirements of the storage system. For example, the preset identifier space reclamation requirement can be set as the requirement to reclaim space optimization at preset intervals, or the preset identifier space reclamation requirement can be triggered when the identifier logical space is full, or the preset identifier space reclamation requirement can be triggered when the preset identifier space is defragmented, or the preset identifier space reclamation requirement can be the requirement to optimize cache disk space, or the preset identifier space reclamation requirement can be the trigger when the key information in the identifier logical space is invalid.

[0252] In this embodiment of the application, the storage system can detect whether the cache disk meets the preset identifier space reclamation requirements. When it is determined that the preset identifier space reclamation requirements are met, it means that the space of the cache disk needs to be optimized, or that the space of the cache disk is at risk of being squeezed. Therefore, the identifier logical space in the cache disk that is full can be filtered out and the filtered identifier logical space can be reclaimed by CG.

[0253] The method described in the above embodiments implements an autonomous GC space reclamation mechanism. This mechanism can more efficiently defragment and release data logical space or identifier logical space, significantly improving reclamation efficiency while greatly reducing the need to reserve new logical space. Furthermore, since this embodiment provides an autonomous GC mechanism that is fully controllable and independent of the data read / write process, the efficiency of user data read / write and bandwidth utilization are significantly improved. Moreover, by eliminating the previously uncontrollable write amplification, the lifespan wear of SSD storage chips becomes relatively controllable, thus improving the accuracy of SSD wear warnings.

[0254] In one embodiment, a method for reclaiming data logical space is provided, namely, "reclaiming data space on the cache disk according to the state of each data logical space in the cache disk" in S1401 above. Figure 20As shown, it includes:

[0255] S1501, determine the status of each data logical space in the cache disk; for the first data logical space in the cache disk that is in use, execute step S1502; for the second data logical space in the cache disk that is full, execute step S1503.

[0256] S1502, No data space reclamation operation is performed on the first data logical space.

[0257] S1503, Based on the effective capacity of the second data logical space, perform data space reclamation on the second data logical space.

[0258] The effective capacity of the second data logical space refers to the amount of valid data contained within it. A high effective capacity indicates a larger amount of invalid data and a smaller amount of valid data, while a low effective capacity indicates a smaller amount of invalid data and a larger amount of valid data. It should be noted that data space reclamation of the second data logical space also refers to the reclamation of the corresponding data block area (BLOCK) of the second data logical space. In this application, logical space reclamation refers to the reclamation or release of the data block area (BLOCK) corresponding to the logical space bucket.

[0259] This application relates to a method for reclaiming data logical space. Specifically, when the storage system filters out a first data logical space in use and a second data logical space in a full state, it is necessary to further detect the effective capacity of the second data logical space. If the effective capacity of the second data logical space is detected to be high, data space reclamation is not performed on the second data logical space. If the effective capacity of the second data logical space is detected to be low, data space reclamation is performed on the second data logical space.

[0260] It should be noted that the above method for reclaiming data space in the second data logical space based on its effective capacity includes: if the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the valid data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released; if the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation is performed on the second data logical space. The third data logical space is an unused logical space, meaning that one or more unused data logical spaces are selected to place the valid data moved from the second data logical space. Afterwards, any remaining invalid data in the original second data logical space can be reclaimed. It should be noted that when processing a full second data logical space, one, two, or more can be processed, and the corresponding requested third data logical spaces can be one or more, without limitation. This involves a method for detecting the effective capacity of the second data logical space, i.e., a method for detecting the fragmentation of the second data logical space. In this embodiment, the validity of each data in the second data logical space can be detected. Specifically, the validity of the data or the fragmentation of the space can be determined by scanning the key information corresponding to each data. When the number of valid data reaches a certain first capacity threshold, or the fragmentation rate reaches a certain threshold, the second data logical space is reclaimed or defragmented. For specific methods of data logical space or fragmentation defragmentation, please refer to [link to relevant documentation]. Figure 21 From left to right, the data logic spaces bucket1 and bucket2, which are in a full state, each contain valid data 'a' and invalid data 'b'. The two full data logic spaces are reclaimed or defragmented, that is, two unused data logic spaces, bucket3 and bucket4, are requested, and the valid data in bucket1 and bucket2 are sequentially written into bucket3 and bucket4. After that, bucket3 goes from an unused state to a full state, and bucket4 goes from an unused state to a used state.

[0261] The method described in the above embodiments achieves autonomous reclamation of the data block area corresponding to the data logical space by operating on the data logical space, avoiding the preemption of user bandwidth. Moreover, the data space reclamation is all equal exchange or the exchange of old space with a small amount of new space, which does not affect the data operation process and requires a small amount of new space, which can reduce the overhead of additional resources and improve the data processing performance of the storage system.

[0262] In one embodiment, an implementation method for reclaiming data logical space is provided, namely, "reclaiming the identifier space of the cache disk according to the state of each identifier logical space in the cache disk" in S1601 above. Figure 22 As shown, it includes:

[0263] S1601, determine the status of each identifier logical space in the cache disk; for the first identifier logical space in the cache disk that is in use, execute step S1602; for the second identifier logical space in the cache disk that is full, execute step S1603.

[0264] S1602, do not perform the identifier space reclamation operation on the first identifier logical space.

[0265] S1603, Based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space, the identifier space is reclaimed.

[0266] The effective capacity of the second identifier logical space refers to the amount of data containing valid key information within it. A high effective capacity indicates that the second identifier logical space contains more invalid information and less valid information, while a low effective capacity indicates that the second identifier logical space contains less invalid information and more valid information.

[0267] This application relates to a method for reclaiming identifier logical space. Specifically, when the storage system filters out a first identifier logical space in use and a second identifier logical space in a full state, it is necessary to further detect the effective capacity of the second identifier logical space. If the effective capacity of the second identifier logical space is high, identifier space reclamation is not performed on the second identifier logical space. If the effective capacity of the second identifier logical space is low, identifier space reclamation can be further performed on the second identifier logical space based on the validity of each piece of information in the second identifier logical space.

[0268] It should be noted that the above method for reclamation of the second identifier logical space based on its effective capacity includes: if the effective capacity of the second identifier logical space is less than the second capacity threshold, further checking the validity of the key information in the second identifier logical space to determine the number of invalid key information; if all the key information in the second identifier logical space is invalid, then directly reclamation of the second identifier logical space; if there is valid key information in the second identifier logical space, then requesting a third identifier logical space, moving the valid key information in the second identifier logical space to the third identifier logical space, and releasing the moved second identifier logical space; if the effective capacity of the second identifier logical space is not less than the second capacity threshold, then no reclamation operation is performed on the second identifier logical space. The third identifier logical space is an unused logical space. That is, one or more unused identifier logical spaces are selected to place valid information moved out of the second identifier logical space. After that, the remaining invalid information in the original second identifier logical space can be reclaimed. It should be noted that when processing a full second identifier logical space, one, two or more can be processed, and the corresponding applied third identifier logical space can be one or more, without limitation.

[0269] In one embodiment, a method for "reclaiming identifier space in the second identifier logical space based on the effective capacity of the second identifier logical space and the validity of key information in the second identifier logical space" as described in S1603 is provided. Figure 23 The method includes:

[0270] S1701, determine the validity of key information in the second identifier logical space. If all key information in the second identifier logical space is invalid, execute step S1702; if there is valid key information in the second identifier logical space, execute step S1703.

[0271] S1702, perform identifier space reclamation on the second identifier logical space.

[0272] S1703, apply for a third identifier logical space, move the valid key information in the second identifier logical space to the third identifier logical space, and release the moved second identifier logical space.

[0273] This application embodiment involves a method for detecting the effective capacity of the second identifier logical space, that is, a method for detecting the fragmentation of the second identifier logical space. In this application embodiment, the validity of each piece of information in the second identifier logical space can be detected. Specifically, the validity of each key piece of information can be determined by identifying the write time, execution information, iteration count, or lock identifier contained in the key information. When it is determined that all key information in the second identifier logical space is invalid, the second identifier logical space can be directly reclaimed. When it is determined that not all key information in the second identifier logical space is invalid, one or more unused third identifier logical spaces can be requested, and then the valid key information in the second identifier logical space can be sequentially moved to the third identifier logical space. For specific methods of identifier logical space or fragmentation management, please refer to [link to relevant documentation]. Figure 24 In this context, from left to right, the identifier logical spaces bucket1 and bucket2 are in a full state. Bucket1 contains valid information 'a' and invalid information 'b', while bucket2 contains only invalid information 'b'. Therefore, during the specific space reclamation, bucket2 can be directly reclaimed, and an identifier logical space bucket3 in an unused state can be requested. The valid information contained in bucket2 can be moved to the identifier logical space bucket3. If the identifier logical space bucket3 is not full after writing, its state changes to the used state. If the identifier logical space bucket3 is full after writing, its state changes to the full state. The corresponding bucket1 changes from a full state to an unused state, and bucket2 changes from a full state to an unused state.

[0274] The method described in the above embodiments provides an autonomous CG space reclamation mechanism. By controlling the operation of logical space buckets, it can be executed during idle periods or when necessary, avoiding bandwidth contention for users. Furthermore, the CG space reclamation mechanism exchanges a majority of old logical space with an equal or small number of new logical space buckets, without affecting data operation processes and requiring very little reserved space; even with only a few buckets, the operation can be performed. Therefore, this method can significantly reduce space overhead and avoid space crowding problems, thereby improving the stability of the entire storage system.

[0275] In one embodiment, considering a scenario where different data block regions within a single cache disk have varying read / write frequencies, resulting in inconsistent wear levels, if a single cache disk experiences severe wear or even fails, the entire cache disk will become unusable. In another scenario, the storage system includes multiple cache disks, each with different read / write frequencies and therefore varying wear levels. Since each cache disk operates relatively independently during data processing, uniform wear across all cache disks would shorten their overall lifespan, thus shortening the overall lifespan of the storage system. Based on these two scenarios, this embodiment introduces a reverse wear mechanism to effectively control the lifespan degradation of SSD disks.

[0276] In one embodiment, the above-mentioned reverse wear method is provided, the method comprising: activating a single-disk reverse wear function to perform granular wear equalization on each cache disk in the storage system when a preset reverse wear requirement is met; and / or activating a multi-disk wear function to perform wear equalization on all cache disks in the storage system.

[0277] Among them, the preset reverse wear requirement is essentially the trigger condition for the reverse wear mechanism. The preset reverse wear requirement can be determined according to the performance requirements of the storage system. For example, the preset reverse wear requirement can be set as the requirement to optimize the cache disk at preset intervals, or the preset reverse wear requirement can be the requirement for a single cache disk to be optimized, or the requirement for all cache disks to be optimized, or the preset reverse wear requirement is the time when the user triggers the wear mechanism for a single cache disk, or the time when the user triggers the wear mechanism for multiple cache disks.

[0278] In this embodiment, the storage system can detect whether a preset reverse wear requirement is met. When it is determined that the preset reverse wear requirement is met, in the following scenarios: First, the cache system can automatically activate the single-disk reverse wear function to perform granular wear equalization on each cache disk, ensuring that the wear degree of all data block areas on each cache disk is consistent or uniform, thus avoiding the entire cache disk becoming unusable due to severe wear of one data block area, thereby wasting resources. Second, the cache system can automatically perform multi-disk reverse wear function to perform granular wear equalization on all cache disks in the storage system, allowing cache disks with high wear to be fully utilized first, extending the lifespan of other cache disks with relatively low wear. Third, the cache system can automatically activate both single-disk and multi-disk reverse wear functions to achieve granular wear equalization on each cache disk and granular uniform wear on multiple disks, ensuring that the wear degree of all data block areas on each cache disk is consistent or uniform, and allowing cache disks with high wear to be fully utilized first. In addition, the aforementioned single-disk reverse wear function or multi-disk reverse wear function can be enabled or disabled by the user. For example, when the cache system is not of high importance or there are other disaster recovery methods available, the reverse wear mechanism can be disabled.

[0279] The method described in the above embodiments introduces an autonomous anti-wear mechanism, which is completely independent of data read and write operations and does not affect each other. It can be turned on or off autonomously, and will not occupy disk space when turned off, thereby improving the lifespan of each cache disk and the overall storage system.

[0280] In one embodiment, a method for implementing the above-mentioned single-disc reverse wear function is provided, such as... Figure 25 As shown, the method includes:

[0281] S1801: For each cache disk, obtain the number of read and write operations for each data block region in the cache disk.

[0282] Each data block region records the number of times it reads and writes data during data processing.

[0283] S1802, configure the enabling priority of each data block area based on the number of reads and writes of each data block area in the cache disk.

[0284] The fewer read and write operations a data block region has, the higher its priority. The priority of a data block region refers to the priority at which it is started. In other words, the priority of a data block region is related to the order in which data blocks are enabled. For example, the higher the priority, the earlier the data block region and its corresponding logical space are started when data is written; the lower the priority, the later the data block region and its corresponding logical space are started when data is written.

[0285] In this embodiment of the application, for a single cache disk SSD in the caching system, when the single-disk reverse wear function is enabled, the activation priority of each data block region can be configured according to the read and write count of each data block region in the cache disk. For example, the activation priority of data block regions with fewer read and write counts can be set to high priority. Thus, when the cache disk has a data write request, when requesting the activation of the corresponding data block region in the logical space, the high-priority data block region can be selected for data processing according to the activation priority of each data block region in the cache disk. This allows data block regions with fewer read and write counts to be processed more, thereby achieving uniform wear of all particles on the entire disk and verifying the lifespan of the cache disk. An exemplary illustration of this method is provided, such as... Figure 26 The diagram shows a comparison of the wear effects of a cache disk using the reverse wear mechanism described in this application's embodiments and one without. The left side shows the remaining read / write cycles for each block in the SSD disk when this solution is not used or reverse wear is disabled, representing the remaining lifespan of each block; the number of read / write cycles is uneven. The right side shows the remaining read / write cycles for each block in the SSD disk when this solution is used and reverse wear is enabled, also representing the remaining lifespan of each block; the number of read / write cycles is uniform.

[0286] The method described in the above embodiments addresses the limitation of the number of read / write cycles for a single block. When a block's read / write cycles are exhausted, it can no longer be written to, leading to damage to the entire SSD disk containing that block. Therefore, uniformizing the granularity of each block on an SSD disk can relatively improve the overall lifespan of the SSD. Furthermore, since the lifespan degradation of SSD storage chips becomes relatively controllable, it can be controlled during logical space allocation, ensuring consistent wear across the entire SSD disk and thus extending its lifespan.

[0287] In one embodiment, a method for implementing the above-mentioned single-disc reverse wear function is provided, such as... Figure 27 As shown, the method includes:

[0288] S1901 determines the remaining lifespan of each cache disk based on the number of read and write operations performed on each cache disk.

[0289] Each data block region records the number of times it is read and written during data processing. The remaining lifespan of the cache disk can be determined based on the data block region with the fewest read and write operations.

[0290] S1902, based on the remaining lifespan of each cache disk, plans the workload of each cache disk.

[0291] The shorter the remaining lifetime of the cache disk, the greater the workload on the cache disk. The workload of the cache disk refers to the probability that the cache disk is assigned a task. That is, the remaining lifetime of the cache disk is related to the probability of the cache disk being assigned a task. For example, the shorter the remaining lifetime, the higher the probability that the cache disk will be assigned a task when writing data; the longer the remaining lifetime, the lower the probability that the cache disk will be assigned a task when writing data.

[0292] In this embodiment, for all SSD cache disks in the caching system, when the multi-disk anti-wear function is activated, the remaining lifespan of each cache disk can be determined first based on the read / write counts of all data block regions in each cache disk. Then, the workload of each cache disk is planned based on its remaining lifespan. For example, the probability of SSD disks with shorter remaining lifespans being assigned tasks is increased to gradient the workload of each SSD disk, making the time when each SSD disk in the storage system reaches the end of its lifespan as distinct as possible, thus preventing all or multiple SSD disks in the caching system from failing simultaneously due to lifespan exhaustion, leading to caching system failure. The method for determining the remaining lifespan of each cache disk based on the read / write counts of all data block regions in each cache disk can be to detect the read / write counts of all data block regions in each cache disk and select the data block region with the fewest remaining read / write counts. This is because the data block region with the fewest remaining read / write counts can represent the remaining lifespan of the entire SSD disk; for example, fewer remaining read / write counts indicate a shorter remaining lifespan. An example illustrating this method is provided below. Figure 28 The diagram shows a comparison of the wear effects of using the reverse wear mechanism described in this application's embodiments and not using the reverse wear mechanism described in this application's embodiments. The top figure shows the remaining lifespan of each SSD in the corresponding cache system when this solution is not used or reverse wear is disabled. Figure 28 (Only four SSDs are shown in the image for illustrative purposes only.) Most of the four SSDs have near-zero remaining lifespan. Below are the remaining lifespans of each SSD in the corresponding cache system when using this solution and enabling reverse wear detection. Figure 28 (Only four SSDs are shown in the image for illustrative purposes only. Most of the four SSDs have a relatively long remaining lifespan.)

[0293] The method described in the above embodiments, because the multiple cache disks in the caching system are independent of each other, differentiates the lifespan of multiple cache disks as much as possible. This avoids the simultaneous failure of all or multiple SSD disks in the caching system due to lifespan exhaustion, leading to caching system malfunction. Therefore, the above method can improve the lifespan of the caching system. Furthermore, because the lifespan of SSD disks is more accurately controlled, the reverse wear operation becomes simpler to implement. More tasks can be allocated to accelerate the wear of SSD disks with shorter lifespans, preventing multiple SSD disks from simultaneously exhausting their lifespan and causing data loss, thus reducing risk.

[0294] In summary, the methods described in all the embodiments offer the following advantages: First, the methods described in the above embodiments utilize the SSD disks in the caching system by directly scheduling the BLOCK through logical space bucket operations, thereby improving the caching system's space management and scheduling capabilities for the SSD disks. Second, the methods described in the above embodiments index and persist the written data using logs, keys, and trees, separating data write and deletion operations. Third, the methods described in the above embodiments process the write operations of logs, keys, data, and other content to disk using a fully sequential write method. Combined with the first and second advantages, this significantly improves the read / write efficiency, bandwidth utilization, and space utilization of the caching system's SSD disks. Fourth, the methods described in the above embodiments allow the caching system to control the lifespan exhaustion time of each SSD disk in the caching system through task scheduling, allowing the more fragile SSD disks to fail first, preventing multiple disks from failing simultaneously due to lifespan exhaustion and causing the caching system to malfunction. In conclusion, the methods described in the embodiments of this application can improve the performance of the storage system.

[0295] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0296] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0297] In one exemplary embodiment, such as Figure 29 As shown, a data processing apparatus is provided, comprising:

[0298] Receiving module 11 is used to receive data processing requests;

[0299] Processing module 12 is used to process data through cache disk and backend disk according to the key information and logical space corresponding to the data processing request; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0300] In an exemplary embodiment, the processing module 12 includes:

[0301] The retrieval unit is used to retrieve the key information from the key information record in memory according to the address index carried in the read request, and obtain the retrieval result;

[0302] The reading unit is used to read data from the cache disk and the back-end disk based on the search results.

[0303] In an exemplary embodiment, the reading unit is specifically configured to read data from the backend disk when the search result is all empty; read data from the cache disk when the search result is all hits; and read the partially hit data from the cache disk and the partially empty data from the backend disk when the search result is partially hits and partially empty.

[0304] In an exemplary embodiment, the processing module 12 includes:

[0305] The first allocation unit is used to allocate a corresponding data logic space for the write request from the data logic space in the use state, and generate key information corresponding to the write request;

[0306] The first write unit is used to write the data to be written in the write request into the data logical space, and to write the data to be written to the back-end disk through the data block area corresponding to the data logical space.

[0307] The second writing unit is used to insert the key information into the key information record in memory, and to write the key information to the cache disk.

[0308] In one exemplary embodiment, the second writing unit described above includes:

[0309] The first allocation subunit is used to allocate a corresponding identification logic space for the key information from the identification logic space of the usage state;

[0310] The first writing subunit is used to write the key information into the identifier logical space and write the key information into the corresponding data block area in the cache disk through the identifier logical space.

[0311] In one exemplary embodiment, the second writing unit described above includes:

[0312] The second allocation subunit is used to allocate corresponding log logic space for the key information from the log logic space in use;

[0313] The second writing subunit is used to write the key information into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, write all the key information in the log logical space into the data block area corresponding to the cache disk.

[0314] In an exemplary embodiment, the second writing subunit is specifically used to determine whether the storage space of the first log logical space in the usage state is sufficient; if the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in the unused state; if the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

[0315] In an exemplary embodiment, the second writing subunit is further configured to determine whether the second log logical space still exists; if the second log logical space exists, the key information is written into the second log logical space; if the second log logical space does not exist, the key information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

[0316] In an exemplary embodiment, the second writing subunit is further configured to generate a log writing request carrying the key information and determine whether the log task has started; if the log task has started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation, and based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in use has sufficient storage space is returned to be executed; if the log task has not started, the log writing request is added to the temporary linked list, and the step of determining whether the log task has started is returned to be executed.

[0317] In an exemplary embodiment, the second writing subunit is further configured to record key information in the memory and write it to the cache disk in units of tree-structured nodes.

[0318] In an exemplary embodiment, the processing module 12 includes:

[0319] The second allocation unit is used to allocate a corresponding data logic space for the write request from the data logic space in the usage state, and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk;

[0320] The processing unit is used to allocate the corresponding data logic space and the target key information according to the write request, and to process the data through the cache disk and the backend disk.

[0321] In an exemplary embodiment, the above-described processing unit is specifically used to insert the invalid key information into the key information record in memory and write it to the cache disk.

[0322] In an exemplary embodiment, the above-mentioned processing unit is specifically used to write the data to be modified by the data modification operation into the data logic space, and to push the data to be modified to the back-end disk through the data block area corresponding to the data logic space.

[0323] The new key information is inserted into the key information record in memory and written to the cache disk.

[0324] In one exemplary embodiment, the first recycling module described above includes:

[0325] The first determining unit is used to determine the state of each data logical space in the cache disk;

[0326] The first recycling unit is configured to not perform data space recycling operations on the first data logical space that is in use in the cache disk.

[0327] The second recycling unit is used to reclaim data space in the second data logical space that is in a full state in the cache disk, based on the effective capacity of the second data logical space.

[0328] In an exemplary embodiment, the second recycling unit is configured to request a third data logical space when the effective capacity of the second data logical space is less than the first capacity threshold, move the effective data in the second data logical space to the third data logical space, and release the moved second data logical space; and not perform data space recycling operation on the second data logical space when the effective capacity of the second data logical space is not less than the first capacity threshold.

[0329] In one exemplary embodiment, the second recycling module described above includes:

[0330] The second determining unit is used to determine the state of each identifier logical space in the cache disk;

[0331] The third recycling unit is used to prevent the first identifier logical space in the cache disk from being recycled.

[0332] The fourth recycling unit is used to reclaim the second identifier logical space in the cache disk that is in a full state, based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

[0333] In an exemplary embodiment, the fourth recycling unit is specifically used to reclaim the second identifier logical space based on the validity of the key information in the second identifier logical space when the effective capacity of the second identifier logical space is less than the second capacity threshold; and not to perform data space reclamation operation on the second identifier logical space when the effective capacity of the second identifier logical space is not less than the second capacity threshold.

[0334] In an exemplary embodiment, the fourth recycling unit is further specifically used to determine the validity of key information in the second identification logical space; if all key information in the second identification logical space is invalid, then the second identification logical space is recycled; if there is valid key information in the second identification logical space, then a third identification logical space is requested, and the valid key information in the second identification logical space is moved to the third identification logical space, and the moved second identification logical space is released.

[0335] In one exemplary embodiment, the data processing apparatus further includes:

[0336] The first initiation wear module is used to initiate the single-disk reverse wear function to perform particle-average wear processing on each cache disk in the storage system when the preset reverse wear requirements are met.

[0337] And / or, a second wear-initiating module is used to initiate the multi-disk wear function to perform uniform wear processing on all cache disks in the storage system.

[0338] In one exemplary embodiment, the first initiation wear module described above includes:

[0339] The acquisition unit is used to acquire the number of read and write operations for each data block region in the cache disk for each cache disk;

[0340] The configuration unit is used to configure the enable priority of each data block region according to the number of reads and writes of each data block region in the cache disk; the fewer the number of reads and writes of a data block region, the higher the priority of the corresponding data block region.

[0341] In one exemplary embodiment, the second initiation wear module described above includes:

[0342] A determining unit is used to determine the remaining lifetime of each cache disk based on the number of read and write operations of each cache disk;

[0343] The planning unit is used to plan the workload of each cache disk based on the remaining lifespan of each cache disk; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

[0344] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0345] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 30 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores and reads / writes data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0346] Those skilled in the art will understand that Figure 30 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0347] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0348] Receive data processing requests;

[0349] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk and a backend disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0350] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0351] Based on the address index carried in the read request, the key information is retrieved from the key information record in memory to obtain the retrieval result;

[0352] Data is read from the cache disk and backend disk based on the search results.

[0353] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0354] If the search result is that all searches are empty, then data is read from the backend disk;

[0355] If the search result is a complete match, then data is read from the cache disk;

[0356] If the search results are partially hit and partially empty, then the partially hit data is read from the cache disk, and the partially empty data is read from the backend disk.

[0357] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0358] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate the key information corresponding to the write request;

[0359] The data to be written in the write request is written to the data logical space, and the data to be written is then pushed to the back-end disk through the data block area corresponding to the data logical space.

[0360] The key information is inserted into the key information record in memory, and the key information is written to the cache disk.

[0361] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0362] From the identification logic space of the usage status, allocate a corresponding identification logic space for the key information;

[0363] The key information is written into the identifier logical space, and then written into the corresponding data block area in the cache disk through the identifier logical space.

[0364] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0365] Allocate corresponding log logic space for the key information from the log logic space in use;

[0366] The key information is written into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, all the key information in the log logical space is written into the data block area corresponding to the cache disk.

[0367] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0368] Determine if the first log logic space for the status in use has sufficient storage space;

[0369] If the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in an unused state;

[0370] If the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

[0371] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0372] Determine whether the second log logic space still exists;

[0373] If the second log logical space exists, the key information is written into the second log logical space;

[0374] If the second log logical space does not exist, the critical information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

[0375] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0376] Generate a log write request carrying the key information and determine whether the log task should be started;

[0377] If it is determined that the log task has been started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation. Based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in the usage state has sufficient storage space is returned to be executed.

[0378] If it is determined that the log task has not been started, the log writing request is added to the temporary list, and the process returns to the step of determining whether the log task has been started.

[0379] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0380] The key information in the memory is recorded and written to the cache disk in a tree-structured node format.

[0381] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0382] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk;

[0383] Based on the write request, allocate the corresponding data logic space and the target key information, and process the data through the cache disk and the backend disk.

[0384] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0385] The invalid key information is inserted into the key information record in memory and written to the cache disk.

[0386] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0387] The data to be modified by the data modification operation is written into the data logic space, and the data to be modified is then written to the back-end disk through the data block area corresponding to the data logic space.

[0388] The new key information is inserted into the key information record in memory and written to the cache disk.

[0389] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0390] When the preset data space reclamation requirements are met, the cache disk is reclaimed according to the state of each data logical space in the cache disk;

[0391] When the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

[0392] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0393] Determine the state of each data logical space in the cache disk;

[0394] For the first data logical space in the cache disk that is in use, no data space reclamation operation is performed on the first data logical space;

[0395] For the second data logical space in the cache disk that is in a full state, data space reclamation is performed on the second data logical space according to the effective capacity of the second data logical space.

[0396] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0397] If the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the effective data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released.

[0398] If the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation will be performed on the second data logical space.

[0399] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0400] Determine the state of each identifier logical space in the cache disk;

[0401] For the first identifier logical space in the cache disk that is in use, no identifier space reclamation operation is performed on the first identifier logical space;

[0402] For the second identifier logical space in the cache disk that is in a full state, the identifier space is reclaimed based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

[0403] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0404] If the effective capacity of the second identifier logical space is less than the second capacity threshold, then the second identifier logical space is reclaimed based on the validity of the key information in the second identifier logical space.

[0405] If the effective capacity of the second identifier logical space is not less than the second capacity threshold, then the data space reclamation operation will not be performed on the second identifier logical space.

[0406] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0407] Determine the validity of key information in the second identifier logical space;

[0408] If all the key information in the second identifier logical space becomes invalid, then the second identifier logical space will be reclaimed.

[0409] If valid key information exists in the second identifier logical space, then a third identifier logical space is requested, and the valid key information in the second identifier logical space is moved to the third identifier logical space, and the moved second identifier logical space is released.

[0410] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0411] Under the condition of meeting the preset reverse wear requirements, the single disk reverse wear function is activated to perform particle-level uniform wear processing on each cache disk in the storage system; and / or, the multi-disk wear function is activated to perform uniform wear processing on all cache disks in the storage system.

[0412] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0413] For each of the cache disks, obtain the number of read and write operations for each data block region in the cache disk;

[0414] The activation priority of each data block region is configured based on the number of reads and writes to each data block region in the cache disk; the fewer the number of reads and writes to a data block region, the higher the priority of the corresponding data block region.

[0415] In one embodiment, the processor, when executing a computer program, also performs the following steps:

[0416] The remaining lifespan of each cache disk is determined based on the number of read and write operations performed on each cache disk.

[0417] The workload of each cache disk is planned based on its remaining lifespan; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

[0418] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0419] Receive data processing requests;

[0420] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk and a backend disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0421] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0422] Based on the address index carried in the read request, the key information is retrieved from the key information record in memory to obtain the retrieval result;

[0423] Data is read from the cache disk and backend disk based on the search results.

[0424] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0425] If the search result is that all searches are empty, then data is read from the backend disk;

[0426] If the search result is a complete match, then data is read from the cache disk;

[0427] If the search results are partially hit and partially empty, then the partially hit data is read from the cache disk, and the partially empty data is read from the backend disk.

[0428] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0429] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate the key information corresponding to the write request;

[0430] The data to be written in the write request is written to the data logical space, and the data to be written is then pushed to the back-end disk through the data block area corresponding to the data logical space.

[0431] The key information is inserted into the key information record in memory, and the key information is written to the cache disk.

[0432] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0433] From the identification logic space of the usage status, allocate a corresponding identification logic space for the key information;

[0434] The key information is written into the identifier logical space, and then written into the corresponding data block area in the cache disk through the identifier logical space.

[0435] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0436] Allocate corresponding log logic space for the key information from the log logic space in use;

[0437] The key information is written into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, all the key information in the log logical space is written into the data block area corresponding to the cache disk.

[0438] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0439] Determine if the first log logic space for the status in use has sufficient storage space;

[0440] If the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in an unused state;

[0441] If the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

[0442] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0443] Determine whether the second log logic space still exists;

[0444] If the second log logical space exists, the key information is written into the second log logical space;

[0445] If the second log logical space does not exist, the critical information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

[0446] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0447] Generate a log write request carrying the key information and determine whether the log task should be started;

[0448] If it is determined that the log task has been started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation. Based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in the usage state has sufficient storage space is returned to be executed.

[0449] If it is determined that the log task has not been started, the log writing request is added to the temporary list, and the process returns to the step of determining whether the log task has been started.

[0450] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0451] The key information in the memory is recorded and written to the cache disk in a tree-structured node format.

[0452] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0453] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk;

[0454] Based on the write request, allocate the corresponding data logic space and the target key information, and process the data through the cache disk and the backend disk.

[0455] From the data logic space in the usage state, allocate a corresponding data logic space for the data deletion operation, and generate invalid key information corresponding to the data deletion operation; the invalid key information includes information about the backend disk;

[0456] The invalid key information is inserted into the key information record in memory and written to the cache disk.

[0457] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0458] The data to be modified by the data modification operation is written into the data logic space, and the data to be modified is then written to the back-end disk through the data block area corresponding to the data logic space.

[0459] The new key information is inserted into the key information record in memory and written to the cache disk.

[0460] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0461] When the preset data space reclamation requirements are met, the cache disk is reclaimed according to the state of each data logical space in the cache disk;

[0462] When the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

[0463] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0464] Determine the state of each data logical space in the cache disk;

[0465] For the first data logical space in the cache disk that is in use, no data space reclamation operation is performed on the first data logical space;

[0466] For the second data logical space in the cache disk that is in a full state, data space reclamation is performed on the second data logical space according to the effective capacity of the second data logical space.

[0467] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0468] If the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the effective data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released.

[0469] If the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation will be performed on the second data logical space.

[0470] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0471] Determine the state of each identifier logical space in the cache disk;

[0472] For the first identifier logical space in the cache disk that is in use, no identifier space reclamation operation is performed on the first identifier logical space;

[0473] For the second identifier logical space in the cache disk that is in a full state, the identifier space is reclaimed based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

[0474] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0475] If the effective capacity of the second identifier logical space is less than the second capacity threshold, then the second identifier logical space is reclaimed based on the validity of the key information in the second identifier logical space.

[0476] If the effective capacity of the second identifier logical space is not less than the second capacity threshold, then the data space reclamation operation will not be performed on the second identifier logical space.

[0477] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0478] Determine the validity of key information in the second identifier logical space;

[0479] If all the key information in the second identifier logical space becomes invalid, then the second identifier logical space will be reclaimed.

[0480] If valid key information exists in the second identifier logical space, then a third identifier logical space is requested, and the valid key information in the second identifier logical space is moved to the third identifier logical space, and the moved second identifier logical space is released.

[0481] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0482] Under the condition of meeting the preset reverse wear requirements, the single disk reverse wear function is activated to perform particle-level uniform wear processing on each cache disk in the storage system; and / or, the multi-disk wear function is activated to perform uniform wear processing on all cache disks in the storage system.

[0483] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0484] For each of the cache disks, obtain the number of read and write operations for each data block region in the cache disk;

[0485] The activation priority of each data block region is configured based on the number of reads and writes to each data block region in the cache disk; the fewer the number of reads and writes to a data block region, the higher the priority of the corresponding data block region.

[0486] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0487] The remaining lifespan of each cache disk is determined based on the number of read and write operations performed on each cache disk.

[0488] The workload of each cache disk is planned based on its remaining lifespan; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

[0489] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0490] Receive data processing requests;

[0491] Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk and a backend disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

[0492] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0493] Based on the address index carried in the read request, the key information is retrieved from the key information record in memory to obtain the retrieval result;

[0494] Data is read from the cache disk and backend disk based on the search results.

[0495] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0496] If the search result is that all searches are empty, then data is read from the backend disk;

[0497] If the search result is a complete match, then data is read from the cache disk;

[0498] If the search results are partially hit and partially empty, then the partially hit data is read from the cache disk, and the partially empty data is read from the backend disk.

[0499] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0500] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate the key information corresponding to the write request;

[0501] The data to be written in the write request is written to the data logical space, and the data to be written is then pushed to the back-end disk through the data block area corresponding to the data logical space.

[0502] The key information is inserted into the key information record in memory, and the key information is written to the cache disk.

[0503] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0504] From the identification logic space of the usage status, allocate a corresponding identification logic space for the key information;

[0505] The key information is written into the identifier logical space, and then written into the corresponding data block area in the cache disk through the identifier logical space.

[0506] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0507] Allocate corresponding log logic space for the key information from the log logic space in use;

[0508] The key information is written into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, all the key information in the log logical space is written into the data block area corresponding to the cache disk.

[0509] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0510] Determine if the first log logic space for the status in use has sufficient storage space;

[0511] If the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in an unused state;

[0512] If the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

[0513] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0514] Determine whether the second log logic space still exists;

[0515] If the second log logical space exists, the key information is written into the second log logical space;

[0516] If the second log logical space does not exist, the critical information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

[0517] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0518] Generate a log write request carrying the key information and determine whether the log task should be started;

[0519] If it is determined that the log task has been started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation. Based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in the usage state has sufficient storage space is returned to be executed.

[0520] If it is determined that the log task has not been started, the log writing request is added to the temporary list, and the process returns to the step of determining whether the log task has been started.

[0521] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0522] The key information in the memory is recorded and written to the cache disk in a tree-structured node format.

[0523] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0524] From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk;

[0525] Based on the write request, allocate the corresponding data logic space and the target key information, and process the data through the cache disk and the backend disk.

[0526] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0527] The invalid key information is inserted into the key information record in memory and written to the cache disk.

[0528] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0529] The data to be modified by the data modification operation is written into the data logic space, and the data to be modified is then written to the back-end disk through the data block area corresponding to the data logic space.

[0530] The new key information is inserted into the key information record in memory and written to the cache disk.

[0531] The new key information is inserted into the key information record in memory and written to the cache disk.

[0532] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0533] When the preset data space reclamation requirements are met, the cache disk is reclaimed according to the state of each data logical space in the cache disk;

[0534] When the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

[0535] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0536] Determine the state of each data logical space in the cache disk;

[0537] For the first data logical space in the cache disk that is in use, no data space reclamation operation is performed on the first data logical space;

[0538] For the second data logical space in the cache disk that is in a full state, data space reclamation is performed on the second data logical space according to the effective capacity of the second data logical space.

[0539] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0540] If the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the effective data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released.

[0541] If the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation will be performed on the second data logical space.

[0542] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0543] Determine the state of each identifier logical space in the cache disk;

[0544] For the first identifier logical space in the cache disk that is in use, no identifier space reclamation operation is performed on the first identifier logical space;

[0545] For the second identifier logical space in the cache disk that is in a full state, the identifier space is reclaimed based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

[0546] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0547] If the effective capacity of the second identifier logical space is less than the second capacity threshold, then the second identifier logical space is reclaimed based on the validity of the key information in the second identifier logical space.

[0548] If the effective capacity of the second identifier logical space is not less than the second capacity threshold, then the data space reclamation operation will not be performed on the second identifier logical space.

[0549] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0550] Determine the validity of key information in the second identifier logical space;

[0551] If all the key information in the second identifier logical space becomes invalid, then the second identifier logical space will be reclaimed.

[0552] If valid key information exists in the second identifier logical space, then a third identifier logical space is requested, and the valid key information in the second identifier logical space is moved to the third identifier logical space, and the moved second identifier logical space is released.

[0553] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0554] Under the condition of meeting the preset reverse wear requirements, the single disk reverse wear function is activated to perform particle-level uniform wear processing on each cache disk in the storage system; and / or, the multi-disk wear function is activated to perform uniform wear processing on all cache disks in the storage system.

[0555] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0556] For each of the cache disks, obtain the number of read and write operations for each data block region in the cache disk;

[0557] The activation priority of each data block region is configured based on the number of reads and writes to each data block region in the cache disk; the fewer the number of reads and writes to a data block region, the higher the priority of the corresponding data block region.

[0558] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:

[0559] The remaining lifespan of each cache disk is determined based on the number of read and write operations performed on each cache disk.

[0560] The workload of each cache disk is planned based on its remaining lifespan; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

[0561] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0562] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0563] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method includes: Receive data processing requests; Based on the key information and logical space corresponding to the data processing request, data processing is performed through a cache disk and a backend disk; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

2. The method according to claim 1, characterized in that, The data processing request is a read request. The data processing, based on the key information and logical space corresponding to the data processing request, is performed through a cache disk and a backend disk, including: Based on the address index carried in the read request, the key information is retrieved from the key information record in memory to obtain the retrieval result; Data is read from the cache disk and backend disk based on the search results.

3. The method according to claim 2, characterized in that, The step of reading data from the cache disk and backend disk based on the search results includes: If the search result is that all searches are empty, then data is read from the backend disk; If the search result is a complete match, then data is read from the cache disk; If the search results are partially hit and partially empty, then the partially hit data is read from the cache disk, and the partially empty data is read from the backend disk.

4. The method according to claim 1, characterized in that, The logical space includes a data logical space, the data processing request is a write request, and the data processing, based on the key information and logical space corresponding to the data processing request, is performed through a cache disk and a backend disk, including: From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate the key information corresponding to the write request; The data to be written in the write request is written to the data logical space, and the data to be written is then pushed to the back-end disk through the data block area corresponding to the data logical space. The key information is inserted into the key information record in memory, and the key information is written to the cache disk.

5. The method according to claim 4, characterized in that, The logical space also includes an identifier logical space, and writing the key information to the cache disk includes: From the identification logic space of the usage status, allocate a corresponding identification logic space for the key information; The key information is written into the identifier logical space, and then written into the corresponding data block area in the cache disk through the identifier logical space.

6. The method according to claim 4, characterized in that, The logical space also includes a log logical space, and writing the key information to the cache disk includes: Allocate corresponding log logic space for the key information from the log logic space in use; The key information is written into the log logical space, and when the key information in the log logical space reaches a preset quantity threshold, all the key information in the log logical space is written into the data block area corresponding to the cache disk.

7. The method according to claim 6, characterized in that, The log logic space is a first log logic space, and writing the key information into the log logic space includes: Determine if the first log logic space for the status in use has sufficient storage space; If the storage space of the first log logical space is insufficient, the key information is written according to the second log logical space in an unused state; If the storage space of the first log logical space is sufficient, the key information is written into the first log logical space.

8. The method according to claim 7, characterized in that, The step of writing the key information into the second log space based on its unused state includes: Determine whether the second log logic space still exists; If the second log logical space exists, the key information is written into the second log logical space; If the second log logical space does not exist, the critical information in the first log logical space is written to the cache disk, and the first log logical space is reclaimed.

9. The method according to claim 7, characterized in that, The method further includes: Generate a log write request carrying the key information and determine whether the log task should be started; If it is determined that the log task has been started, the log writing request is added to the processing linked list, and log writing requests of a preset capacity are extracted from the processing linked list for aggregation. Based on the key information corresponding to the aggregation result, the step of determining whether the first log logical space in the usage state has sufficient storage space is returned to be executed. If it is determined that the log task has not been started, the log writing request is added to the temporary list, and the process returns to the step of determining whether the log task has been started.

10. The method according to claim 4, characterized in that, The step of writing the key information to the cache disk includes: Key information in memory is recorded and written to the cache disk in a tree-structured manner, node by node.

11. The method according to claim 1, characterized in that, The logical space includes a data logical space. The data processing request is a write request, which indicates a data deletion operation or a data modification operation. The data processing, based on the key information and logical space corresponding to the data processing request, is performed through a cache disk and a backend disk, including: From the data logic space in the usage state, allocate a corresponding data logic space for the write request and generate target key information corresponding to the write request; the target key information includes invalid key information or new key information; the invalid key information includes the information of the backend disk, and the new key information includes the information of the backend disk and the information of the new cache disk; Based on the write request, allocate the corresponding data logic space and the target key information, and process the data through the cache disk and backend disk.

12. The method according to claim 11, characterized in that, The write request indicates a data deletion operation, the target key information includes the invalid key information, and the process of allocating corresponding data logical space and the target key information according to the write request, and processing the data through a cache disk, includes: The invalid key information is inserted into the key information record in memory and written to the cache disk.

13. The method according to claim 11, characterized in that, The write request indicates a data modification operation, the target key information includes the new key information, and the process of allocating corresponding data logical space and the target key information according to the write request, and processing the data through a cache disk, includes: The data to be modified by the data modification operation is written into the data logic space, and the data to be modified is then written to the back-end disk through the data block area corresponding to the data logic space. The new key information is inserted into the key information record in memory and written to the cache disk.

14. The method according to any one of claims 1-13, characterized in that, The method further includes: When the preset data space reclamation requirements are met, the cache disk is reclaimed according to the state of each data logical space in the cache disk; When the preset identifier space reclamation requirement is met, the identifier space of the cache disk is reclaimed according to the state of each identifier logical space in the cache disk and the validity of the key information in each identifier logical space.

15. The method according to claim 14, characterized in that, The method based on each of the cache disks The state of the data logical space, and the data space reclamation of the cache disk, including: Determine the state of each data logical space in the cache disk; For the first data logical space in the cache disk that is in use, no data space reclamation operation is performed on the first data logical space; For the second data logical space in the cache disk that is in a full state, data space reclamation is performed on the second data logical space according to the effective capacity of the second data logical space.

16. The method according to claim 15, characterized in that, The step of reclaiming data space in the second data logical space based on its effective capacity includes: If the effective capacity of the second data logical space is less than the first capacity threshold, then a third data logical space is requested, and the effective data in the second data logical space is moved to the third data logical space, and the moved second data logical space is released. If the effective capacity of the second data logical space is not less than the first capacity threshold, then no data space reclamation operation will be performed on the second data logical space.

17. The method according to claim 14, characterized in that, The step of reclaiming identifier space in the cache disk based on the state of each identifier logical space in the cache disk and the validity of key information in each identifier logical space includes: Determine the state of each identifier logical space in the cache disk; For the first identifier logical space in the cache disk that is in use, no identifier space reclamation operation is performed on the first identifier logical space; For the second identifier logical space in the cache disk that is in a full state, the identifier space is reclaimed based on the effective capacity of the second identifier logical space and the validity of the key information in the second identifier logical space.

18. The method according to claim 17, characterized in that, The step of reclaiming data space in the second identifier logical space based on its effective capacity and the validity of key information within the second identifier logical space includes: If the effective capacity of the second identifier logical space is less than the second capacity threshold, then the second identifier logical space is reclaimed based on the validity of the key information in the second identifier logical space. If the effective capacity of the second identifier logical space is not less than the second capacity threshold, then no data space reclamation operation will be performed on the second identifier logical space.

19. The method according to claim 18, characterized in that, The step of reclaiming the identifier space based on the validity of key information in the second identifier logical space includes: Determine the validity of key information in the second identifier logical space; If all the key information in the second identifier logical space becomes invalid, then the second identifier logical space will be reclaimed. If valid key information exists in the second identifier logical space, then a third identifier logical space is requested, and the valid key information in the second identifier logical space is moved to the third identifier logical space, and the moved second identifier logical space is released.

20. The method according to any one of claims 1-13, characterized in that, The method further includes: Under the condition of meeting the preset reverse wear requirements, the single disk reverse wear function is activated to perform particle-level uniform wear processing on each cache disk in the storage system; and / or, the multi-disk wear function is activated to perform uniform wear processing on all cache disks in the storage system.

21. The method according to claim 20, characterized in that, The single-disk reverse wear function performs granular wear equalization on each cache disk in the storage system, including: For each of the cache disks, obtain the number of read and write operations for each data block region in the cache disk; The activation priority of each data block region is configured based on the number of reads and writes to each data block region in the cache disk; the fewer the number of reads and writes to a data block region, the higher the priority of the corresponding data block region.

22. The method according to claim 20, characterized in that, The multi-disk wear reduction function performs a uniform wear reduction process on all cache disks in the storage system, including: The remaining lifespan of each cache disk is determined based on the number of read and write operations performed on each cache disk. The workload of each cache disk is planned based on its remaining lifespan; the shorter the remaining lifespan of each cache disk, the greater the workload of the corresponding cache disk.

23. A data processing apparatus, characterized in that, The device includes: The receiving module is used to receive data processing requests; The processing module is used to process data through a cache disk and a backend disk based on the key information and logical space corresponding to the data processing request; the logical space corresponds to the data block area in the cache disk; the key information includes the mapping relationship between the cache disk and the backend disk.

24. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 22.

25. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 22.

26. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 22.

Images