Data processing method and apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose data processing methods, apparatuses, devices and storage media; can be applied to archival storage, a data deletion request for a tape library can be obtained, the tape library stores at least one data block, the data block is encoded into at least one write data unit, each data write unit has a corresponding unit state, the data block is generated by at least one storage data unit aggregation, and the data deletion request indicates a target data block to be deleted; according to the data deletion request, storage data units of the target data block are identified as to-be-deleted storage data units; when the identification result indicates that all storage data units of the target data block are to-be-deleted storage data units, the unit state of the write data unit of the target data block is updated; and based on the state update result of each write data unit of the target data block, data deletion processing is performed on each write data unit of the target data block. The scheme can improve the data processing efficiency and data management effect of the tape library.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] Magnetic tape, as a storage medium with advantages such as large data storage capacity, long media preservation time, and low cost, can effectively address the storage challenges brought about by rapidly growing data storage demands. A magnetic tape library is a hardware container that manages multiple magnetic tapes and enables reading and writing of data within the tapes.

[0003] In the process of researching and practicing related technologies, the inventors of this application discovered that when data deletion is performed on the stored data of a tape library, if the tape library is temporarily unavailable, such as due to a malfunction or firmware upgrade, other data deletion tasks in the system will be blocked, or related data deletion will fail and remain in the tape library, forming junk data that is difficult to clean up. This will undoubtedly significantly reduce the data processing efficiency and data management effect of the tape library. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, and storage medium that can improve the efficiency of data processing for tape libraries.

[0005] This application provides a data processing method, including:

[0006] Obtain a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each of the data write units has a corresponding unit state, the data block is generated by aggregating at least one storage data unit, and the data deletion request indicates the target data block to be deleted.

[0007] Based on the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and the identification result is obtained.

[0008] When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit status of the write data units of the target data block is updated;

[0009] Based on the status update results of each written data unit of the target data block, data deletion processing is performed on each written data unit of the target data block.

[0010] Accordingly, embodiments of this application also provide a data processing apparatus, including:

[0011] An acquisition unit is used to acquire a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each of the data write units has a corresponding unit state, the data block is generated by aggregating at least one stored data unit, and the data deletion request indicates the target data block to be deleted.

[0012] The identification unit is used to identify the storage data units to be deleted in the target data block according to the data deletion request, and obtain the identification result;

[0013] An update unit is used to update the cell status of the write data cells of the target data block when the identification result indicates that all storage data cells in the target data block are storage data cells to be deleted.

[0014] The first deletion unit is used to perform data deletion processing on each written data unit of the target data block based on the status update results of each written data unit of the target data block.

[0015] In one embodiment, the first deletion unit includes:

[0016] The first acquisition subunit is used to acquire the status update results of each written data unit of the target data block;

[0017] The first generation subunit is used to generate a data deletion task for the target data block when the status update result indicates that all write data units of the target data block have been successfully updated;

[0018] The first deletion subunit is used to perform data deletion processing on each written data unit of the target data block according to the data deletion task.

[0019] In one embodiment, the first deletion unit further includes:

[0020] The first determining subunit is used to determine the update time for updating the unit status of the target data block when the status update result indicates that the update of part of the written data unit of the target data block has failed.

[0021] A status update subunit is used to update the unit status of the write data unit of the target data block at the update time.

[0022] The second deletion subunit is used to perform data deletion processing on each written data unit of the target data block based on the status update results of each written data unit of the target data block.

[0023] In one embodiment, the acquisition unit includes:

[0024] The second acquisition subunit is used to acquire data deletion requests for the target storage data unit;

[0025] The second determining subunit is used to determine the target data block that aggregates the target storage data unit from the stored data of the tape library;

[0026] The first detection subunit is used to detect data deletion requests in each storage data unit in the target data block and obtain the detection results.

[0027] The second generation subunit is used to generate a data deletion request for the target data block when the detection result indicates that all storage data units in the target data block have corresponding data detection requests.

[0028] In one embodiment, the acquisition unit includes:

[0029] The third acquisition subunit is used to acquire a data migration request for the tape library, wherein the data migration request indicates data migration of a target data block, and each storage data unit in the target data block is a storage data unit to be migrated.

[0030] The third determining subunit is used to determine the target tape library required for data migration of the target data block;

[0031] The first migration subunit is used to migrate the target data block from the tape library currently storing the target data block to the target tape library;

[0032] The third generation subunit is used to generate a data deletion request for the target data block when it is detected that the data migration of the target data block is completed.

[0033] In one embodiment, the first migration subunit is configured to:

[0034] Each write data unit of the target data block is read from the tape library currently storing the target data block into the tape library buffer; data verification is performed on each write data unit to obtain a verification result; when the verification result indicates that the verification is successful, each write data unit of the target data block is migrated from the tape buffer to the target tape library.

[0035] In one embodiment, the data processing apparatus further includes:

[0036] The determining unit is configured to determine, when the identification result indicates that some storage data units in the target data block are storage data units to be retained, a first tape library currently storing the target data block and a second tape library required for data migration of the target data block;

[0037] A migration unit is used to migrate the storage data units to be retained in the target data block from the first tape library to the second tape library;

[0038] The second deletion unit is used to perform data deletion processing on each write data unit of the target data block stored in the first tape library when the migration is detected to be complete.

[0039] In one embodiment, the migration unit includes:

[0040] The second migration subunit is used to migrate each write data unit of the target data block from the first tape library to the tape library buffer.

[0041] The verification subunit is used to verify the data in each of the written data units and obtain the verification result.

[0042] The third migration subunit is used to migrate the storage data units to be retained in the target data block from the tape library buffer to the second tape library when the verification result indicates that the verification is passed.

[0043] In one embodiment, the third migration subunit is configured to:

[0044] Obtain a set of storage data units required for data migration of storage data units, wherein the set of storage data units includes at least one storage data unit to be migrated; aggregate the storage data units in the set of storage data units and the storage data units to be retained in the target data block to obtain at least one aggregated data block; migrate the aggregated data block from the tape library buffer to the second tape library.

[0045] In one embodiment, the third migration subunit is specifically used for:

[0046] The storage data unit set is updated based on the storage data units to be retained in the target data block; the data capacity of the updated storage data unit set is calculated based on the data volume information of each storage data unit in the updated storage data unit set; when the data capacity is greater than a preset capacity threshold, the storage data units in the updated storage data unit set are aggregated.

[0047] Accordingly, this application also provides a computer device including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any of the data processing methods provided in this application.

[0048] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing a computer program, which is loaded by a processor to execute any of the data processing methods provided in embodiments of this application.

[0049] Accordingly, this application also provides a computer program product, including a computer program / instructions, wherein when the computer program / instructions are executed by a processor, they implement the steps of the data processing method shown in the embodiments of this application.

[0050] This application embodiment can obtain a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit stored in the tape library, each write data unit has a corresponding unit state, the data block is generated by aggregating at least one storage data unit, and the data deletion request indicates a target data block to be deleted; according to the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and an identification result is obtained; when the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit state of the write data units of the target data block is updated; based on the state update result of each write data unit of the target data block, data deletion processing is performed on each write data unit of the target data block.

[0051] In this scheme, at least one storage data unit can be aggregated to generate a data block, and the storage data unit can be written into the tape library in units of data blocks. Specifically, the data block can be encoded into at least one write data unit and stored in the tape library. For example, each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0052] When all storage data units in the target data block to be deleted are storage data units to be deleted, this scheme updates the unit status of each write data unit in the target data block, such as updating the unit status of each write data unit in the target data block to a marked-for-deletion state. Based on the status update results of each write data unit in the target data block, for example, when the unit status of each write data unit in the target data block has been successfully updated, indicating that all tape libraries are reachable, the data is deleted from each write data unit in the target data block. In this way, through a two-stage deletion scheme of status update and status detection, it is ensured that even if some tape libraries become unavailable, each write data unit in the target data block can still be deleted successfully, thereby avoiding the generation of difficult-to-clean garbage data and improving the data management effect of tape libraries. Furthermore, because the two-stage deletion scheme in this approach sets the first stage as a lightweight state update operation rather than a heavyweight data deletion operation, it can buy time for the system to assess risks when some tape libraries become unavailable. This allows the system to reallocate computing resources to other data deletion tasks when a partial tape library becomes unavailable, rather than being blocked on the current data deletion task. This effectively reduces the probability of system blocking and improves the data processing efficiency of the tape library. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating a scenario of the method provided in an embodiment of this application;

[0055] Figure 2 This is a flowchart of the data processing method provided in the embodiments of this application;

[0056] Figure 3 This is a schematic diagram of the software architecture of the data processing method provided in the embodiments of this application;

[0057] Figure 4 This is a schematic diagram of the unit state update of the data processing method provided in the embodiments of this application;

[0058] Figure 5 This is another schematic flowchart of the data processing method provided in the embodiments of this application;

[0059] Figure 6 This is a schematic diagram of the structure of the data processing apparatus provided in the embodiments of this application;

[0060] Figure 7 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0061] Figure 8 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0062] Figure 9 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0063] Figure 10 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0064] Figure 11 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0065] Figure 12 This is another structural schematic diagram of the data processing apparatus provided in the embodiments of this application;

[0066] Figure 13 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. However, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0068] This application provides a data processing method that can be executed by a computer device. The computer device can be a terminal or a server; that is, the data processing method can be executed by a terminal or a server, or by both a terminal and a server.

[0069] Specifically, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application does not impose any restrictions.

[0070] In one embodiment, such as Figure 1As shown, server 10 can obtain data deletion requests for a tape library, wherein the tape library can store at least one data block, the data block is encoded as at least one write data unit stored in the tape library, each write data unit has a corresponding unit state, and the data block is generated by aggregating at least one stored data unit. The data deletion request indicates the target data block to be deleted. For example, server 10 can integrate a data storage engine capable of interacting with the tape library hardware, such as an ultra-cold data storage engine, and a user can initiate a data deletion request for the tape library through terminal 20, so that the data storage engine on server 10 can obtain the data deletion request. For example, the user can initiate the data deletion request through the Web or an Application Programming Interface (API).

[0071] Server 10 can identify the storage data units to be deleted in the target data block according to the data deletion request, and obtain the identification result. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit status of the written data units in the target data block is updated. Based on the status update results of each written data unit in the target data block, data deletion processing is performed on each written data unit in the target data block. Optionally, in practical applications, server 10 can also send a processing result message to terminal 20 so that the user can know the data deletion result.

[0072] The following will provide a detailed description of each example. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0073] The data processing method provided in this application can be executed by a server or a terminal, or by both a server and a terminal. This application example illustrates the data processing method executed by a server.

[0074] like Figure 2 The specific process of this data processing method can be described as follows:

[0075] 101. Obtain a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each write data unit has a corresponding unit state, the data block is generated by aggregating at least one stored data unit, and the data deletion request indicates the target data block to be deleted.

[0076] Among them, magnetic tape is a low-cost storage medium; a magnetic tape library is a hardware container that manages multiple magnetic tapes and can read and write data on the magnetic tapes.

[0077] A data block (or block in English) refers to a large, continuous segment of data. Specifically, when writing data to a tape library, it can be done in units of data blocks.

[0078] Here, a write data unit refers to the basic unit when writing data to a tape library. Specifically, in this application, a data block can be encoded as at least one write data unit and stored in the tape library. For example, a data block can be encoded as at least one write data unit, and each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0079] There are various ways to implement encoding. For example, it can be based on data segmentation, encoding a data block into at least one writeable data unit. For instance, error-correcting coding (EC) can be used to encode a data block into at least one writeable data unit. Specifically, EC is a data protection method that segments data into fragments, expands and encodes redundant data blocks, and stores them in different locations, such as disks, storage nodes, tape libraries, or other geographical locations.

[0080] As an implementation of EC encoding, when using EC to encode data, the data can be divided into K fragments, and (NK) parity fragments can be generated. There are a total of N data fragments and parity fragments. Therefore, this implementation of EC encoding can be simply referred to as KN. For example, the EC mechanism can be used to encode a data block as at least one write data unit. Specifically, when using the KN EC mechanism to encode a data block, a data block can be encoded as N write data units, where K are data fragments and (NK) are parity fragments.

[0081] It is worth noting that when using the KN-style EC mechanism to encode data, in addition to the concept of "shards," the concept of "columns" can also be used. That is, the encoded data can include multiple data columns, where K columns are data columns and (NK) columns are check columns, for a total of N columns. As an example, a Shard can be used to refer to the data unit being written. When using the KN-style EC mechanism to encode a block, each encoded column is a Shard. That is, when the block is finally written to tape, it will consist of N Shards, where K are data Shards and (NK) are check Shards.

[0082] The unit state of the data writing unit refers to the state of the data writing unit during the process of writing data into the tape library. For example, the unit state of the data writing unit can include: temporary state, steady state, and label / delete state.

[0083] In this context, a storage data unit refers to the basic unit of stored data that the tape library interacts with external systems. For example, a server may integrate a data storage engine capable of interacting with the tape library hardware. In this case, the storage data unit can be considered a concept at the data storage engine layer. For instance, a user can send a data storage request to the server, specifying the file that the user needs to store; this file can then serve as a storage data unit.

[0084] As an example, when applied to archive storage, the specific storage data unit can be a DeepArchive Blob (Dab), where Blob is an abbreviation for Binary Large Object. A binary large object is a data type that can store binary objects or data. Specifically, in a database, binary large objects can be used to store binary data, such as images, multimedia files, and executable software code.

[0085] In this application, a data block can be generated by aggregating at least one storage data unit. Therefore, the size of the data block can be related to the data size of the at least one storage data unit, that is, the size of the data block does not have to be fixed.

[0086] Furthermore, to improve the data processing efficiency of tape libraries, thresholds related to data block size can be set. For example, a lower limit threshold for data block size can be set. In practical applications, performance evaluation of the tape library can be performed. It might be found that when the file size written to the tape library is less than or equal to X, it significantly impacts the tape library's performance, causing performance degradation or degradation to an unacceptable degree. Conversely, when the file size written to the tape library is greater than X, it does not significantly impact performance, causing no performance degradation or degradation to be within an acceptable range. Therefore, the threshold X obtained from the performance evaluation of the tape library can be determined as the lower limit threshold for data block size; that is, the data block size can be set to be greater than this threshold X.

[0087] Since a data block can be generated by aggregating at least one storage data unit, and the size of the data block can be set to be greater than a threshold X, a data block can be determined based on the storage data unit when the data size of the storage data unit is greater than the threshold X; when the data size of the storage data unit is not greater than the threshold X, multiple storage data units can be aggregated to generate a data block, so that the data size of the data block generated by aggregating multiple storage data units is greater than the threshold X.

[0088] There are several ways to aggregate at least one storage data unit. For example, at least one storage data unit can be spliced ​​together to aggregate them into a data block, or at least one storage data unit can be encoded to aggregate them into a data block, and so on.

[0089] The data deletion request is used to request the deletion of data stored in the tape library. In this application, the data deletion request can be used to indicate the target data block that needs to be deleted.

[0090] For example, a data deletion request can be sent by a user requesting the deletion of data from a storage unit in the tape library. In this case, the data block containing that storage unit can be identified as the target data block to be deleted. Similarly, a data deletion request can be sent by the data storage engine's operations engineer requesting the deletion of data from a data block in the tape library. In this case, the data block indicated by the data deletion request can be identified as the target data block to be deleted.

[0091] It is worth noting that when a data deletion request does not directly specify the target data block to be deleted, such as when the data deletion request specifies a tape library, write data unit, or storage data unit that needs to be deleted, the target data block to be deleted can be determined based on the relationship between the tape library, data block, write data unit, and storage data unit. That is, the tape library stores at least one data block, the data block is encoded as at least one write data unit stored in the tape library, each write data unit has a corresponding unit state, and the data block is generated by aggregating at least one storage data unit.

[0092] As an example, one could consider an ultra-cold data storage engine used for archival storage, where the server integrates a data interaction engine capable of interacting with tape library hardware. See [link to relevant documentation]. Figure 3 Users can initiate data deletion requests to the Cloud Object Storage (COS) service via the web or API. COS is a distributed storage service for storing massive amounts of files. Correspondingly, COS can send the received data deletion request to the access layer of the ultra-cold data storage engine, triggering the access layer to add a data deletion task based on the received data deletion request. This allows the access layer to subsequently distribute the data deletion task to the worker nodes in the ultra-cold data storage engine, so that the worker nodes can execute the specific data deletion task.

[0093] In one embodiment, considering that data blocks can be used not only as units when writing data to tape, but also specifically as units when performing data deletion operations, and since a data block is generated by aggregating at least one storage data unit, when a user's data deletion request for a target storage data unit is received, in order to prevent accidental data deletion, for a target data block in the tape library that aggregates the target storage data unit, the server can be configured to only further generate a data deletion request for the target data block when all storage data units of the target data block are storage data units to be deleted, so as to facilitate subsequent data deletion processing of the target data block. Specifically, the step "obtaining a data deletion request for the tape library" may include:

[0094] Retrieve the data deletion request for the target storage data unit;

[0095] Identify the target data block containing the target storage data unit from the stored data of the tape library;

[0096] Data deletion request detection is performed on each storage data unit in the target data block to obtain the detection results;

[0097] When the detection results indicate that all storage data units in the target data block have corresponding data detection requests, a data deletion request for the target data block is generated.

[0098] Specifically, a data deletion request for a target storage data unit is used to request the deletion of data from that target storage data unit. For example, if the storage data unit is a file, when a user wants to delete a target file stored in a tape library, the user can initiate a data deletion request for that target file via the Web or API, so that the server can receive the data deletion request for the target file.

[0099] Furthermore, the server can identify the data block containing the target storage data unit from the stored data of the tape library; this data block is the target data block.

[0100] For example, when aggregating at least one storage data unit to generate a data block, the server can record the aggregation correspondence between the data block and the storage data unit. That is, it records which storage data units were aggregated to generate the data block, or in other words, which data block each storage data unit is aggregated into. In this way, the server can determine the target data block with the target storage data unit aggregated from the data blocks stored in the tape library based on the aggregation correspondence between the data block and the storage data unit.

[0101] Specifically, detecting data deletion requests for storage data units can be used to determine whether there is a data deletion request for that storage data unit; in other words, it can be used to determine whether the storage data unit is a storage data unit to be deleted. Similarly, the server can determine the storage data units that aggregate to generate the target data block based on the aggregation correspondence between data blocks and storage data units, and then perform data deletion request detection on the determined storage data units.

[0102] For example, a server can integrate a storage system for caching data deletion requests. Specifically, when a data deletion request for a stored data unit is received, the server can cache that request. Therefore, for each stored data unit that aggregates to generate the target data block, the server can check whether there is a data deletion request for it in the storage system to detect such requests and obtain the detection result for each stored data unit. Specifically, if the detection result for a stored data unit indicates that there is a data deletion request for that stored data unit in the storage system, then that stored data unit can be determined as a stored data unit to be deleted.

[0103] If the detection results for each storage data unit that aggregates to generate the target data block all indicate that there is a corresponding data deletion request, for example, if all indicate that there is a data deletion request for that storage data unit in the storage system, the server can determine that each storage data unit that aggregates to generate the target data block is a storage data unit to be deleted. Therefore, the server can generate a data deletion request for the target data block so that the data deletion process can be carried out on the target data block in the future.

[0104] As an example, see Figure 3In practical applications, users can initiate data deletion requests for storage data units to COS via the Web or API. COS can then send these requests to the access layer of the ultra-cold data storage engine. Upon receiving the request, the access layer temporarily stores it. When the server receives the user's data deletion request for the target storage data unit, it can identify the target data block containing that data unit from the tape library's stored data and perform data deletion request checks on each storage data unit that generated the target data block. Specifically, if the access layer detects that all storage data units that generated the target data block have collected data deletion requests for them, the server can determine that all storage data units that generated the target data block are storage data units to be deleted, and can then generate a data deletion request for that target data block. Optionally, a data deletion task for the target data block can be generated based on the data deletion requests and distributed to each worker node.

[0105] In one embodiment, it is considered that in addition to the generation of a data block deletion request being triggered by a data deletion request for a storage data unit (for example, the generation of a data block deletion request being triggered by a user's data deletion request for a target storage data unit, where the target storage data unit refers to the storage data unit to be deleted specified by the user through the data deletion request, and the target data block refers to the data block that aggregates the target storage data unit), there may be other situations that trigger the generation of a data block deletion request, such as when space reclamation is required for the tape library.

[0106] Specifically, the following section will introduce the space reclamation for tape libraries.

[0107] There are various scenarios where tape library space reclamation is required. For example, since tape media does not support overwrite, in applications where data blocks stored on tape need to be deleted, the resulting gaps cannot be directly utilized. Instead, defragmentation and data migration are necessary before the deleted space can be reused. Therefore, to improve storage space reuse during data deletion and better reclaim reusable storage space, a data deletion command can be sent to the source tape library after the data block is migrated from its source to the target tape library, triggering the deletion process. In other words, business-side data deletion requirements for tape libraries can necessitate tape library space reclamation.

[0108] In addition, in practical applications, other situations may trigger the need to reclaim space for tape libraries. For example, space reclamation may be required for tape libraries due to maintenance or upgrades, or for the management of stored data.

[0109] Meanwhile, due to the ineradicable nature of tape data, the ultimate effect of reclaiming space in a tape library is to migrate data blocks stored on the source tape to a target tape, which could be a new blank tape distinct from the source tape. Once data is successfully written to the target tape, data deletion is triggered on the source tape. Specifically, after all written data units on the source tape are deleted, the source tape is formatted, becoming a blank tape. This allows the storage space on the blank tape to be reused, effectively reclaiming the storage space on the source tape.

[0110] Therefore, it can be seen that by performing data migration processing on the tape library to achieve space reclamation of the tape library, a data deletion request for the tape library can be triggered. In one embodiment, taking the data deletion request for the tape library from the business side as an example, which leads to the need for space reclamation of the tape library, the step "obtaining the data deletion request for the tape library" can specifically include:

[0111] Obtain a data migration request for the tape library, wherein the data migration request indicates that data should be migrated to the target data block, and each storage data unit in the target data block is a storage data unit to be migrated.

[0112] Identify the target tape library required for data migration of the target data block;

[0113] Migrate the target data block from the current tape library storing the target data block to the target tape library;

[0114] When the data migration to the target data block is detected to be complete, a data deletion request for the target data block is generated.

[0115] Specifically, a data migration request for a tape library can be used to request data migration processing of data blocks stored in the tape library. Therefore, the data block is the target data block to be migrated as indicated by the data migration request.

[0116] It is worth noting that in this application, since a data block can be generated by aggregating at least one storage data unit, the target data block to be migrated can also be generated by aggregating at least one storage data unit. Furthermore, based on the data migration situation of the storage data units that aggregate to generate the target data block, the target data block to be migrated can be divided into the following two cases:

[0117] 1.1 All storage data units that are aggregated to generate the target data block are storage data units that are to be migrated.

[0118] 1.2 Not all of the storage data units in the aggregated target data block are storage data units to be migrated. In other words, some of the storage data units are storage data units that will not be migrated, i.e., storage data units to be retained.

[0119] Here we can discuss the target data block under case 1.1, that is, the data migration of the target data block in response to the data migration request instruction of the tape library. In this case, each storage data unit in the target data block is a storage data unit to be migrated.

[0120] There are several situations that can trigger the generation of a data migration request for a tape library. For example, when space reclamation of a tape library is required, a data migration request for the tape library can be generated. For details, please refer to the above, which will not be repeated here.

[0121] For example, for a magnetic tape with space to be reclaimed, the tape library can be requested to obtain a list of data blocks stored on that tape using the tape's identification information. This identification information could include, for example, the tape's serial number (IdentityDocument, ID) and product serial number (SN). Further, a target data block can be selected from the data block list and added to the migration task list. All storage data units that aggregate to generate the target data block are considered storage data units to be migrated. Each migration task in the migration task list is determined by a data migration request, and each data migration request instructs the migration of a target data block. In this way, the server can obtain data migration requests for the tape library through the migration task list.

[0122] Due to the ineradicable nature of tape data, the ultimate effect of migrating a target data block is to move the target data block stored on the source tape to the target tape. For example, the target tape could be a new blank tape distinct from the source tape. Therefore, the tape library containing the target tape can be defined as the target tape library required for migrating the target data block. Optionally, the target tape library can be set to be a tape library distinct from the source tape library, where the source tape library is the tape library containing the source tape.

[0123] Then, the target data block can be migrated from the current tape library storing the target data block to the target tape library. For example, the target data block can be read from the current tape library and written to the target tape library. In this way, when the data migration of the target data block is detected to be complete, a data deletion request for the target data block can be generated to send a data deletion instruction for the data block to the source tape library, triggering the data deletion process for the data block.

[0124] In one embodiment, considering the data read / write characteristics of magnetic tape, a tape library buffer can be set up to assist in migrating the target data block from the tape library currently storing the target data block to the target tape library. Specifically, the step "migrating the target data block from the tape library currently storing the target data block to the target tape library" may include:

[0125] Each write data unit of the target data block is read from the tape library currently storing the target data block into the tape library buffer;

[0126] Perform data verification on each written data unit to obtain the verification result;

[0127] When the verification result indicates that the verification passed, the write data units of the target data block are migrated from the tape buffer to the target tape library.

[0128] Due to the data read / write characteristics of magnetic tape, which means that magnetic tape can be considered a non-standard storage device, it is impossible to directly obtain data from the magnetic tape in practical applications. Therefore, a tape library buffer can be set up to read data from the magnetic tape into the tape library buffer so that data can be read normally through the tape library buffer. Here, the tape library buffer can be understood as a standard conventional storage device.

[0129] Specifically, the process of reading data stored in the source tape into the tape library buffer can also be called data warm-up. In this application, data warm-up can be performed on the target data block at the granularity of the written data unit. After the target data block is warmed up, the data will be read from the source tape into the tape library buffer. In practical applications, for each tape in the tape library, it can be processed one tape at a time to perform data warm-up on each written data unit stored on the tape.

[0130] Data verification aims to detect or verify relevant data to avoid errors that may occur after data transmission or storage.

[0131] There are several ways to implement data verification for each written data unit. For example, if the EC mechanism is used to encode the target data block as at least one written data unit, then data verification is performed on each written data unit of the target data block. Specifically, this can be achieved by verifying the Cyclic Redundancy Check (CRC) code of each written data unit. If a potential silent error is found that causes the data comparison of some written data units of the target data block to fail, the data repair process can be initiated. Specifically, the remaining written data units of the target data block can be read, and the missing data can be calculated to achieve data repair.

[0132] For example, data verification can be performed on each written data unit based on a parity check mechanism; or data verification can be performed on each written data unit based on an accumulation check mechanism; and so on.

[0133] When the verification result indicates that all write data units of the target data block have passed the verification, it can be determined that no errors have occurred due to data transmission or saving. Therefore, the data still in the tape library buffer, that is, the write data units of the target data block, can be directly written from the tape buffer to the target tape library, so as to realize the migration of the write data units of the target data block from the tape buffer to the target tape library.

[0134] Optionally, for application scenarios where data deletion requests from the business side necessitate tape library space reclamation, after each write data unit of the target data block is successfully written from the tape buffer to the target tape library, a data deletion command for the target data block is sent to the source tape library storing the target data block to trigger data deletion processing. Furthermore, after all write data units stored in the source tape library have been deleted, a formatting command can be executed on the source tape library to release storage space.

[0135] 102. Based on the data deletion request, identify the storage data units to be deleted in the target data block and obtain the identification results.

[0136] Among them, identifying storage data units to be deleted can be used to determine whether a storage data unit is a storage data unit to be deleted.

[0137] Specifically, the data deletion request here indicates the target data block to be deleted. In this application, a data block can be generated by aggregating at least one storage data unit. Therefore, the target data block to be deleted can also be generated by aggregating at least one storage data unit. Further, based on the data deletion status of the storage data unit that generated the target data block, the target data block to be deleted can be divided into the following two cases:

[0138] 2.1 All storage data units that are aggregated to generate the target data block are storage data units to be deleted.

[0139] 2.2 Among all the storage data units that are aggregated to generate the target data block, not all of them are storage data units to be deleted. In other words, some of the storage data units are storage data units that are not to be deleted, that is, storage data units to be retained.

[0140] In practical applications, since the data deletion request does not necessarily specify the target data block to be deleted directly, but can specify the tape library, write data unit, or storage data unit that needs to be deleted, the storage data unit to be deleted can be identified based on the relationship between the tape library, data block, write data unit, and storage data unit. That is, the tape library stores at least one data block, the data block is encoded as at least one write data unit stored in the tape library, each write data unit has a corresponding unit state, and the data block is generated by aggregating at least one storage data unit.

[0141] For example, if the data deletion request directly specifies the target data block that needs to be deleted, then all storage data units in the target data block, that is, all storage data units that are aggregated to generate the target data block, are storage data units to be deleted.

[0142] For example, if the data deletion request directly specifies a tape library that needs to be deleted, and the target data block is stored in that tape library, then all storage data units in the target data block, that is, all storage data units that aggregate to generate the target data block, are storage data units to be deleted.

[0143] For example, if the data deletion request directly specifies a write data unit that needs to be deleted, and the write data unit is a write data unit generated by encoding the target data block, then it can be determined that all storage data units in the target data block, that is, all storage data units that aggregate to generate the target data block, are storage data units to be deleted.

[0144] For example, if the data deletion request directly specifies a storage data unit that needs to be deleted, and that storage data unit belongs to the storage data units that were aggregated to generate the target data block, then that storage data unit can be determined as the storage data unit to be deleted. Furthermore, in this case, if the server can integrate a storage system for caching data deletion requests, this storage system can be used to perform data deletion request detection on each storage data unit that was aggregated to generate the target data block, in order to determine whether that storage data unit is the storage data unit to be deleted.

[0145] 103. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, update the cell status of the write data units of the target data block.

[0146] If the identification result indicates that all storage data units in the target data block are storage data units to be deleted, then it corresponds to the relevant description in situation 2.1 above.

[0147] The unit state of a data writing unit refers to the state of the data writing unit during the process of writing data to the tape library. For example, the unit state of a data writing unit can include: a temporary state, a stable state, and a mark-to-delete state. The temporary state refers to the existence state of the data writing unit when it is written to the buffer; the stable state refers to the existence state of the data writing unit when the writing to the buffer is complete; and the mark-to-delete state refers to the existence state of the data writing unit after it has been written to the buffer and is marked as data to be deleted.

[0148] There are several ways to represent the state of a data writing unit. For example, considering that the data writing unit is a file and its writing process in the tape library is a non-atomic operation, the filename can be used to represent the state of the data writing unit. Specifically, when writing to the buffer, the file exists in a temporary state, and the corresponding filename can be set to end with .tmp; when the file writing to the buffer is completely completed, the file exists in a stable state, and the corresponding filename can be set to end with .done; and when the file is marked as a file to be deleted, the file ends in a marked-delete state, and the corresponding filename can be set to end with .delete.

[0149] In addition, in practical applications, the unit status of the data writing unit can be presented by methods such as creating relevant tables in the database to record the unit status of the data writing unit, or creating corresponding unit status variables for each data writing unit to record the unit status of the data writing unit.

[0150] In this application, the cell state of the write data unit of the target data block is updated, specifically, the write data unit can be updated from the current cell state to the mark-delete state.

[0151] It is worth noting that although the cell status of the data writing unit can be presented in various ways, there are also multiple ways to update the cell status of the data writing unit. For example, you can change the file name of the data writing unit to end with ".delete", or change the relevant table entry in the database to the value representing the "delete" state, or assign the cell status variable of the data writing unit to the value representing the "delete" state, etc. However, it should be noted that no matter which specific implementation method is used, it is a lightweight operation compared to the operation of data deletion.

[0152] As an example, when the identification result indicates that all storage data units in the target data block are storage data units to be deleted, a Shard can be used to refer to the write data unit. Let's take the target data block as being encoded into six write data units: Shard1, Shard2, Shard3, Shard4, Shard5, and Shard6. Specifically, when all write data units of the target data block have been written to the cache, it is known that the unit states of Shard1 to Shard6 are all in a steady state. Further attempts can be made to update the unit states of these Shards from the steady state to the delete state, thereby updating the unit states of each write data unit in the target data block.

[0153] 104. Based on the status update results of each written data unit of the target data block, perform data deletion processing on each written data unit of the target data block.

[0154] Specifically, in step 103, the result of updating the cell state of each write data unit in the target data block can include update success and update failure. Update success indicates that the write data unit was successfully updated from its current cell state to the marked / deleted state, while update failure indicates that the write data unit failed to be updated from its current cell state to the marked / deleted state.

[0155] For example, you can use the example above, see [link to example]. Figure 4During the process of updating the cell states of Shard1 to Shard6 of the target data block, the state update results of Shard1, Shard2 and Shard4 were successful and all were marked as tombstones, while the state update results of Shard3, Shard5 and Shard6 were unsuccessful and none of them were marked as tombstones.

[0156] For ease of understanding, we can define a written data unit as a "tombstone" when it is successfully updated to the marked-for-delete state. Therefore, when any written data unit in the target data block is successfully updated to the marked-for-delete state, it can be assumed that a deletion action has been triggered, i.e., a tombstone exists, and thus the data deletion phase can begin. It is worth noting that to avoid missing data deletions, we can configure the process to confirm that the status update results of all written data units in the target data block are successful before performing data deletion. Specifically, the step "Based on the status update results of each written data unit in the target data block, perform data deletion processing on each written data unit of the target data block" can include:

[0157] Obtain the status update results of each written data unit of the target data block;

[0158] When the status update result indicates that all write data units of the target data block have been successfully updated, a data deletion task for the target data block is generated.

[0159] Based on the data deletion task, data deletion processing is performed on each written data unit of the target data block.

[0160] As an example, see Figure 4 When the status update results of Shard1 to Shard6 are all successful, that is, Shard1 to Shard6 are all marked as tombstones, a data deletion task for the target data block can be generated so that data deletion processing of Shard1 to Shard6 can be performed according to the data deletion task.

[0161] As another example, see Figure 3Once a data deletion task targeting the data block is generated, it can be distributed to each worker node of the ultra-cold data storage engine. The worker nodes then perform data deletion on the write data units of the target data block stored in the tape library. In practical applications, this storage engine can also include a central management node. The central management node can interact with the worker nodes and the tape library group. For example, worker nodes can request storage space allocation from the central management node and submit storage space requests. The central management node and the tape library group can update each other with relevant storage space information. A tape library group is a set of hardware containers that manage multiple tapes, i.e., a set of tape libraries. Figure 3 The image shows two tape library groups, each containing j tape libraries.

[0162] In one embodiment, considering that each write data unit of the target data block can be stored in different tape libraries, if the tape library storing the write data unit becomes temporarily unavailable when updating the status of a certain write data unit, the status update result of the write data unit will fail. That is, the status update result of some write data units of the target data block will fail. If a deletion operation is still performed on the target data under such circumstances, the write data units stored in the temporarily unavailable tape library will not be deleted due to the temporary unavailability of the tape library. Subsequently, when the tape library is restored, the write data units will become difficult-to-clean garbage data on the tape library (because it is difficult to determine what kind of data this part is, for example, it is difficult to determine whether this part of the data belongs to the data that was not completely deleted during the previous data deletion, or the residual data left over after some data was lost due to media problems).

[0163] Therefore, to avoid generating difficult-to-clean garbage data, after updating the cell status of the write data units of the target data block, if the update result of some write data units fails, a data deletion task for the target data block can be generated only when the update results of all write data units are successful. Based on this data deletion task, data deletion processing is performed on each write data unit of the target data block. Specifically, the data processing method may also include:

[0164] When the status update result indicates that the update of part of the write data unit of the target data block has failed, determine the update time for updating the unit status of the target data block;

[0165] The cell status of the write data unit of the target data block is updated at the update time;

[0166] Based on the status update results of each written data unit of the target data block, data deletion processing is performed on each written data unit of the target data block.

[0167] Here, the update time refers to the time required to update the cell status of the write data unit of the target data block again. For example, the update time can be set to the time after a preset waiting period from the current time. Specifically, this preset waiting period represents the waiting time for the current data deletion task by the system.

[0168] In practical applications, there are several ways to determine the preset waiting time. For example, the number of failed write data units can be determined, and the larger the number, the longer the waiting time can be set. Alternatively, the number of failed write data unit updates can be obtained, and the larger the number, the longer the waiting time can be set. Another example is that it can be set in combination with business needs and resource considerations.

[0169] As an example, see Figure 4 After the initial cell state update of each shard in the target data block, only some shards are marked as tombstones, indicating a successful update. Therefore, the server can determine the update time for the next cell state update of the target data block and attempt to update all shards of the target data block to the marked-to-delete state at that time. Furthermore, once all shards of the target data block are in the marked-to-delete state, data deletion can be performed on each shard of the target data block.

[0170] As mentioned above, updating the state of written data units is a lightweight operation compared to data deletion. In this application, the data deletion process for each written data unit of the target data block is implemented through a two-stage deletion scheme: a lightweight state update operation in the first stage and a data deletion operation in the second stage. Therefore, if it is found in the first stage that some written data units cannot even be updated by the lightweight state update operation, it is possible to know at a very low cost that the tape library containing these written data units is currently unavailable. In this way, the system can flexibly schedule computing resources, for example, by relinquishing computing resources to execute other tasks, thereby avoiding task blocking or the generation of garbage data that is difficult to clean up.

[0171] In contrast, if a partial tape library becomes unavailable only during a heavyweight data deletion operation, requiring all write data units to be successfully deleted will force the data deletion task to be retried until success, during which time all other tasks in the task queue will be blocked. However, if partial write data unit deletion failures are allowed, write data units stored in temporarily unavailable tape libraries will be unable to be deleted due to the temporary unavailability of the tape library. Subsequently, when the tape library is restored, these write data units will become difficult-to-clean garbage data on the tape library.

[0172] If, based on the status update results of each write data unit of the target data block, it is determined that some write data units of the target data block have failed to update, then it can be determined that part of the tape library is in an unavailable state. Therefore, the update time for updating the unit status of the target data block again can be determined so that the system can try to update the unit status of the write data units of the target data block again at that update time, so as to ensure that data deletion processing will only be performed on each write data unit of the target data block after all write data units of the target data block are in a marked-delete state.

[0173] As can be seen, in the process of deleting target data blocks based on the two-phase deletion scheme, if some related tape libraries become unavailable, this can be detected in the first phase through a lightweight state update operation. Furthermore, the inclusion of a waiting period for state update retry prevents the system from immediately retrying, thus delaying the originally system-blocking action at a relatively low cost. If the lightweight state update operation confirms that all related tape libraries are currently available in the first phase, it strongly guarantees the successful execution of the heavyweight data deletion operation in the second phase. Therefore, the two-phase deletion mechanism significantly reduces the probability of system blockage and allows for timely and flexible scheduling of system resources when some related tape libraries are unavailable, minimizing the negative impact of hardware failures on system operation.

[0174] In one embodiment, considering the identification of storage data units to be deleted in the target data block, the identification result can indicate that all storage data units in the target data block are storage data units to be deleted, and can also indicate that some storage data units in the target data block are storage data units to be retained, which corresponds to the relevant description in the aforementioned situation 2.2. In this case, data deletion processing of the target data block can be achieved by migrating only the storage data units to be retained in the target data block. It is worth noting that since this data processing method can improve the storage space reuse rate of the data migration process by reclaiming reusable storage space after migrating the target data block, this data processing method is also applicable to the target data block in the aforementioned situation 1.2. Specifically, the data processing method may further include:

[0175] When the identification result indicates that some storage data units in the target data block are storage data units to be retained, determine the first tape library for storing the current target data block and the second tape library required for data migration of the target data block;

[0176] The storage data units to be retained in the target data block are migrated from the first tape library to the second tape library;

[0177] When migration is detected as complete, data deletion is performed on each write data unit of the target data block stored in the first tape library.

[0178] For example, if not all of the storage data units in the aggregated target data block are storage data units to be deleted, in other words, some of the storage data units are storage data units that are not to be deleted, then these storage data units that are not to be deleted can be used as storage data units to be retained in the target data block.

[0179] For example, if not all of the storage data units in the aggregated target data block are storage data units to be migrated, in other words, some of the storage data units are storage data units that are not to be migrated, then these storage data units that are not to be migrated can be used as storage data units to be retained in the target data block.

[0180] Here, the first tape library refers to the tape library currently storing the target data block, and the second tape library refers to the tape library used to store the storage data units to be retained in the target data block after the data migration is completed. Specifically, the data migration of the storage data units to be retained in the target data block can be achieved by migrating the storage data units to be retained in the target data block from the first tape library to the second tape library.

[0181] As an example, for a tape with space to be reclaimed, the tape library can be requested to obtain a list of data blocks stored on the tape using the tape's identification information. Data blocks meeting preset migration conditions can be selected from this list and added to a migration data block list. This migration data block list includes at least one data block that meets the preset migration conditions. For instance, the target data block in scenarios 1.2 and 2.2 can be considered a data block that meets the preset migration conditions. Therefore, when the identification result indicates that some storage data units in the target data block are storage data units to be retained, the target data block can be added to the migration data block list. Subsequently, the server can migrate the storage data units to be retained from the target data block in the migration data block list from the first tape library to the second tape library.

[0182] Furthermore, when it is detected that the migration of the storage data units to be retained in the target data block from the first tape library to the second tape library is completed, the storage space in the first tape library can be reclaimed by performing data deletion processing on each written data unit of the target data block stored in the first tape library, so as to improve the storage space reuse rate.

[0183] In one embodiment, considering the data read / write characteristics of magnetic tape, a tape library buffer can be set up to assist in migrating the storage data units to be retained in the target data block from the first tape library to the second tape library. Specifically, the step of "migrating the storage data units to be retained in the target data block from the first tape library to the second tape library" may include:

[0184] Each write data unit of the target data block is moved from the first tape library to the tape library buffer;

[0185] Perform data verification on each written data unit to obtain the verification result;

[0186] When the verification result indicates that the verification passed, the storage data units to be retained in the target data block are migrated from the tape library buffer to the second tape library.

[0187] Specifically, the explanation for the step "migrate each write data unit of the target data block from the first tape library to the tape library buffer; perform data verification on each write data unit to obtain the verification result" can be found in the explanation for the aforementioned step "read each write data unit of the target data block from the tape library currently storing the target data block to the tape library buffer; perform data verification on each write data unit to obtain the verification result," with the difference being:

[0188] 1) Here, the starting tape library for data migration is set to the first tape library for storing the target data block, while in the previous steps it was the tape library for storing the target data block.

[0189] 2) In the aforementioned steps, when performing data warm-up on the target data block, the granularity can be based on the written data unit, while here it is based on the data block. For example, in practical applications, data warm-up can be performed on each data block in the migration data block list at the data block level. After the data block is warmed up, the data will be read from the first tape library corresponding to each data block into the tape library buffer.

[0190] Furthermore, when the verification result indicates that all written data units of the target data block have passed the verification, it can be determined that no errors have occurred due to data transmission or saving. Therefore, the storage data units to be retained in the target data block can be directly written from the tape buffer to the second tape library from the data still in the tape library buffer, so as to realize the migration of the storage data units to be retained in the target data block from the tape library buffer to the second tape library.

[0191] In one embodiment, considering that only a portion of the storage data units in the aggregated target data block are storage data units to be retained, these storage data units to be retained can be re-aggregated into a new data block so that each storage data unit in the aggregated data block is a storage data unit to be retained, thereby facilitating system data management. Specifically, the step "migrating the storage data units to be retained in the target data block from the tape library buffer to the second tape library" may include:

[0192] Obtain the set of storage data units required for data migration of storage data units, wherein the set of storage data units includes at least one storage data unit to be migrated;

[0193] Aggregate the storage data units in the storage data unit set and the storage data units to be retained in the target data block to obtain at least one aggregated data block;

[0194] The aggregated data blocks are moved from the tape library buffer to the second tape library.

[0195] The storage data unit set refers to a set consisting of at least one storage data unit to be migrated.

[0196] For example, a list of migration data blocks can be obtained, and each data block in the list can be checked. Specifically, when checking a single data block, assuming that the data block has N storage data units, of which M are not to be retained, the ID information and data size information of the remaining NM storage data units to be retained can be recorded, and these storage data units to be retained can be added to a storage data unit set. In this way, based on the data blocks in the migration data block list, the set of storage data units required for data migration of the storage data units can be obtained.

[0197] Furthermore, considering that if non-retainable storage data units are removed from all storage data units of the target data block, it may not be possible to aggregate and generate a new data block based solely on the retainable storage data units in the target data block. For example, if the data block generated by aggregating only these storage data units is no larger than a threshold X, then the retainable storage data units in the target data block can be aggregated using the storage data units in the storage data unit set to generate a new data block, i.e., the aggregated data block.

[0198] In this way, by migrating the aggregated data block from the tape library buffer to the second tape library, the storage data units to be retained in the target data block can be migrated from the tape library buffer to the second tape library. It is worth noting that in practical applications, after the data migration is successful—that is, after the data is successfully written to the new tape—the data block containing the stored data unit will change. Therefore, this change can be communicated to upstream businesses so that the system can better manage data. Furthermore, after the data migration is successful, the target data block stored in the first tape library can be deleted.

[0199] As an example, if the implementation method for determining the storage data unit set is based on the data blocks in the migration data block list, specifically by checking all data blocks in the migration data block list, determining the storage data units to be retained for each data block, and then generating a storage data unit set composed of these retained storage data units, then it can be seen that the storage data unit set obtained by the server is static and unchanging data. Therefore, at least one aggregated data block can be generated by repeating the following operation: selecting the storage data units to be aggregated from the storage data unit set, aggregating these storage data units, and the data size of the generated aggregated data block is greater than the threshold X. Optionally, after the data migration is successful, that is, after the data is successfully written to the new tape, data deletion can be performed on the original data blocks corresponding to the re-aggregated storage data units in the first tape library.

[0200] As another example, if the implementation method for determining the storage data unit set based on the data blocks in the migration data block list is as follows: During the inspection of each data block in the migration data block list, whenever a storage data unit to be retained is determined for a data block, this storage data unit to be retained is added to the storage data unit set. That is, the storage data unit set obtained by the server is dynamically updated. Therefore, when it is detected that the storage data units in this dynamically updated storage data unit set meet the requirements for aggregation to generate a data block, the aggregated data block is generated by aggregating these storage data units. Since the storage data unit set is continuously dynamically updated in this example, new aggregated data blocks can be continuously generated as the storage data unit set is dynamically updated. Specifically, the step "aggregating the storage data units in the storage data unit set and the storage data units to be retained in the target data block" can include:

[0201] Update the set of storage data units based on the storage data units to be retained in the target data block;

[0202] Calculate the data capacity of the updated storage data unit set based on the data volume information of each storage data unit in the updated storage data unit set.

[0203] When the data capacity exceeds the preset capacity threshold, the storage data units in the updated storage data unit set will be aggregated.

[0204] The data volume information of the storage data unit is used to indicate the data size of the storage data unit.

[0205] The data capacity of the storage data unit set is used to indicate the total data size of all storage data units in the storage data unit set.

[0206] During the continuous dynamic updating of the storage data unit set, when it is detected that the data capacity of the updated storage data unit set is greater than a preset capacity threshold, for example, greater than threshold X, the storage data units in the updated storage data unit set can be aggregated to obtain an aggregated data block generated by these storage data units.

[0207] Similarly, once the data migration is successful, that is, once the data is successfully written to the new tape, the original data block corresponding to the re-aggregated storage data unit in the first tape library can be deleted.

[0208] As can be seen from the above, this embodiment can obtain a data deletion request for a tape library. The tape library stores at least one data block, which is encoded as at least one write data unit and stored in the tape library. Each write data unit has a corresponding unit state. The data block is generated by aggregating at least one storage data unit. The data deletion request indicates the target data block to be deleted. According to the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and an identification result is obtained. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit state of the write data units of the target data block is updated. Based on the state update results of each write data unit of the target data block, data deletion processing is performed on each write data unit of the target data block.

[0209] In this scheme, at least one storage data unit can be aggregated to generate a data block, and the storage data unit can be written into the tape library in units of data blocks. Specifically, the data block can be encoded into at least one write data unit and stored in the tape library. For example, each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0210] When all storage data units in the target data block to be deleted are storage data units to be deleted, this scheme updates the unit status of each write data unit in the target data block, such as updating the unit status of each write data unit in the target data block to a marked-for-deletion state. Based on the status update results of each write data unit in the target data block, for example, when the unit status of each write data unit in the target data block has been successfully updated, indicating that all tape libraries are reachable, the data is deleted from each write data unit in the target data block. In this way, through a two-stage deletion scheme of status update and status detection, it is ensured that even if some tape libraries become unavailable, each write data unit in the target data block can still be deleted successfully, thereby avoiding the generation of difficult-to-clean garbage data and improving the data management effect of tape libraries.

[0211] Furthermore, because the two-stage deletion scheme in this approach sets the first stage as a lightweight state update operation rather than a heavyweight data deletion operation, it can buy time for the system to assess risks when some tape libraries become unavailable. This allows the system to reallocate computing resources to other data deletion tasks when a partial tape library becomes unavailable, rather than being blocked on the current data deletion task. This effectively reduces the probability of system blocking and improves the data processing efficiency of the tape library.

[0212] Based on the method described in the above embodiments, the following examples will provide further detailed explanations.

[0213] In this embodiment, the data processing device will be integrated into the server as an example for explanation. Figure 5 As shown, a data processing method has the following specific steps:

[0214] 201. The server obtains a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit, stored in the tape library, each write data unit has a corresponding unit state, the data block is generated by aggregating at least one stored data unit, and the data deletion request indicates the target data block to be deleted.

[0215] For example, a server can integrate an ultra-cold data storage engine. Its software architecture diagram can be found in Figure 3. A Block can refer to a data block, a Shard to a data writing unit, a ShardStatus to the status of the data writing unit, and a Dab to a data storage unit. Furthermore, the ultra-cold data storage engine can be defined to use a KN-based EC mechanism, encoding a Block into N Shards and writing them to N different tape libraries. Therefore, when a Block needs to be deleted, all N corresponding Shards must be deleted.

[0216] For example, when writing data, the worker nodes in the ultra-cold data storage engine can write data to the tape library at the block level. A block can be sliced ​​according to EC (Extended EC encoding), and the sliced ​​data becomes a shard. Taking KN EC encoding as an example, a block can first be divided into K equal-sized data shards, and then (NK) columns of check shards are calculated. A total of N shards need to be written to the tape library, and these N shards will be written to N different tape libraries for disaster recovery.

[0217] As a file, a shard is written to in a tape library in a non-atomic process. Specifically, when writing to the buffer, the file exists in a temporary state (the filename ends with .tmp); when the data writing to the buffer is completely completed, the file is closed and becomes stable (the filename is changed to end with .done); while shards marked for deletion end with .delete.

[0218] As an example, users can initiate data deletion requests to COS via the web or API, and COS can send the received data deletion requests to the access layer of the cold data storage engine to trigger the access layer to add a new data deletion task based on the received data deletion request.

[0219] 202. Based on the data deletion request, the server identifies the storage data units to be deleted in the target data block and obtains the identification results.

[0220] As an example, the access layer of the ultra-cold data storage engine can temporarily store the received data deletion requests. In this way, if the access layer collects all the data deletion requests for all the storage data units in the target data block, it can be determined that all the storage data units in the target data block are storage data units to be deleted; if the access layer does not collect all the data deletion requests for all the storage data units in the target data block, it can be determined that some of the storage data units in the target data block are storage data units to be retained.

[0221] 203. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the server updates the unit status of the write data units in the target data block.

[0222] For example, you can try changing all Shards of the target data block from a stable state (filenames ending in .done) to a marked-delete state (i.e., ShardStatus = kMarkDelete, and filenames ending in .delete) to update the cell state of each Shard in the target data block. For ease of understanding, a Shard in the kMarkDelete state can be called a "tombstone".

[0223] 204. Based on the status update results of each written data unit of the target data block, the server performs data deletion processing on each written data unit of the target data block.

[0224] For example, if any shard of the target data block is marked as kMarkDelete, it can be determined that a user has triggered a deletion action (tombstone exists), and the data deletion phase can begin. Before performing data deletion, it is necessary to forcibly check whether all shards of the target data block are in the kMarkDelete state. If not all shards are currently in the kMarkDelete state, a round of marking all shards can be performed first, and data deletion can be performed after all shards are successfully marked. If marking all shards still fails, it proves that some shards are still unreachable (which may generate garbage data that is difficult to clean up). In this case, the entire task can be set to continue retrying until the condition of all shards being in the kMarkDelete state is met before data deletion is performed.

[0225] 205. When the identification result indicates that some storage data units in the target data block are storage data units to be retained, the server determines the first tape library currently storing the target data block and the second tape library required for data migration of the target data block.

[0226] 206. The server migrates the storage data units to be retained in the target data block from the first tape library to the second tape library.

[0227] For example, a migration data block list (BlockList) can be obtained, and the target data block can be added to BlockList. BlockList includes at least one data block that meets preset migration conditions. Furthermore, data in BlockList can be warmed up at the block level. Taking the target data block as an example, after warming up, the data is read from the source tape storing the target data block into the tape library buffer. During data reading, the CRC of the entire data can be checked for verification. If a potential silent error is found causing data comparison failure, a data repair process is initiated. For example, if data comparison failure is found in some shards of the target data block, the remaining shards of the target data block can be read, and the missing data can be calculated to achieve data repair.

[0228] Furthermore, the inspection can begin from the first block in the BlockList. Specifically, taking the target data block as an example, assuming the target data block contains N dabs, of which M are deleted, the DabId and corresponding Dab size of the remaining NM dabs can be recorded, and these dabs can be added to the storage data unit set DabList. During the dynamic updating of DabList, when the total size of the dabs in DabList exceeds the block packing threshold X, these dabs can be repacked into a new block and written to a new destination tape. For the target data block, this destination tape can specifically be a new destination tape in the second tape library. In addition, the blocks involved in these repacked dabs in the source tape can be deleted. For example, after the data migration for the target data block is successful, data deletion processing can be performed on the target data block stored in the first tape library.

[0229] It is worth noting that after a new block is successfully written to a new tape, the filename of the block containing the Dab in that new block will change, so the DabId also needs to be changed. At this time, the upstream business can be notified of the change in DabId, for example, by informing both the old DabId and the new DabId.

[0230] 207. When the migration is detected to be complete, the server performs data deletion processing on each write data unit of the target data block stored in the first tape library.

[0231] As can be seen from the above, in the embodiments of this application, at least one storage data unit can be aggregated to generate a data block, and the storage data unit can be written into the tape library in units of data blocks. Specifically, in this scheme, the data block can be encoded into at least one write data unit and stored in the tape library. For example, each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0232] When all storage data units in the target data block to be deleted are storage data units to be deleted, this scheme updates the unit status of each write data unit in the target data block, such as updating the unit status of each write data unit in the target data block to a marked-for-deletion state. Based on the status update results of each write data unit in the target data block, for example, when the unit status of each write data unit in the target data block has been successfully updated, indicating that all tape libraries are reachable, the data is deleted from each write data unit in the target data block. In this way, through a two-stage deletion scheme of status update and status detection, it is ensured that even if some tape libraries become unavailable, each write data unit in the target data block can still be deleted successfully, thereby avoiding the generation of difficult-to-clean garbage data and improving the data management effect of tape libraries. Furthermore, because the two-stage deletion scheme in this approach sets the first stage as a lightweight state update operation rather than a heavyweight data deletion operation, it can buy time for the system to assess risks when some tape libraries become unavailable. This allows the system to reallocate computing resources to other data deletion tasks when a partial tape library becomes unavailable, rather than being blocked on the current data deletion task. This effectively reduces the probability of system blocking and improves the data processing efficiency of the tape library.

[0233] In one embodiment, a data processing apparatus is provided, such as Figure 6 As shown, the data processing device may include: an acquisition unit 301, an identification unit 302, an update unit 303, and a first deletion unit 304, as follows:

[0234] The acquisition unit 301 can be used to acquire a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each of the data write units has a corresponding unit state, the data block is generated by aggregating at least one stored data unit, and the data deletion request indicates the target data block to be deleted.

[0235] The identification unit 302 can be used to identify the storage data units to be deleted in the target data block according to the data deletion request, and obtain the identification result;

[0236] The update unit 303 can be used to update the cell status of the write data units of the target data block when the identification result indicates that all storage data units in the target data block are storage data units to be deleted.

[0237] The first deletion unit 304 can be used to perform data deletion processing on each written data unit of the target data block based on the status update results of each written data unit of the target data block.

[0238] In one embodiment, reference Figure 7 The first deletion unit 304 may include:

[0239] The first acquisition subunit 3041 can be used to acquire the status update results of each written data unit of the target data block;

[0240] The first generation subunit 3042 can be used to generate a data deletion task for the target data block when the status update result indicates that all write data units of the target data block have been successfully updated;

[0241] The first deletion subunit 3043 can be used to perform data deletion processing on each written data unit of the target data block according to the data deletion task.

[0242] In one embodiment, reference Figure 8 The first deletion unit 304 may further include:

[0243] The first determining subunit 3044 can be used to determine the update time for updating the unit status of the target data block when the status update result indicates that the partial write data unit update of the target data block has failed.

[0244] The status update subunit 3045 can be used to update the cell status of the write data unit of the target data block at the update time.

[0245] The second deletion subunit 3046 can be used to perform data deletion processing on each written data unit of the target data block based on the status update results of each written data unit of the target data block.

[0246] In one embodiment, reference Figure 9 The acquisition unit 301 may include:

[0247] The second acquisition subunit 3011 can be used to acquire a data deletion request for a target storage data unit;

[0248] The second determining subunit 3012 can be used to determine the target data block that aggregates the target storage data unit from the stored data of the tape library;

[0249] The first detection subunit 3013 can be used to detect data deletion requests in each storage data unit in the target data block and obtain detection results.

[0250] The second generation subunit 3014 can be used to generate a data deletion request for the target data block when the detection result indicates that all storage data units in the target data block have corresponding data detection requests.

[0251] In one embodiment, reference Figure 10 The acquisition unit 301 may include:

[0252] The third acquisition subunit 3015 can be used to acquire a data migration request for a tape library, wherein the data migration request indicates data migration of a target data block, and each storage data unit in the target data block is a storage data unit to be migrated.

[0253] The third determining subunit 3016 can be used to determine the target tape library required for data migration of the target data block;

[0254] The first migration subunit 3017 can be used to migrate the target data block from the tape library currently storing the target data block to the target tape library;

[0255] The third generation subunit 3018 can be used to generate a data deletion request for the target data block when it is detected that the data migration of the target data block is completed.

[0256] In one embodiment, the first migration subunit 3017 can be used to:

[0257] Each write data unit of the target data block is read from the tape library currently storing the target data block into the tape library buffer; data verification is performed on each write data unit to obtain a verification result; when the verification result indicates that the verification is successful, each write data unit of the target data block is migrated from the tape buffer to the target tape library.

[0258] In one embodiment, reference Figure 11 The data processing device may further include:

[0259] The determining unit 305 can be used to determine the first tape library currently storing the target data block and the second tape library required for data migration of the target data block when the identification result indicates that some storage data units in the target data block are storage data units to be retained;

[0260] Migration unit 306 can be used to migrate the storage data units to be retained in the target data block from the first tape library to the second tape library;

[0261] The second deletion unit 307 can be used to perform data deletion processing on each write data unit of the target data block stored in the first tape library when the migration is detected to be complete.

[0262] In one embodiment, reference Figure 12 The migration unit 306 may include:

[0263] The second migration subunit 3061 can be used to migrate each write data unit of the target data block from the first tape library to the tape library buffer.

[0264] The verification subunit 3062 can be used to perform data verification on each of the written data units and obtain the verification result.

[0265] The third migration subunit 3063 can be used to migrate the storage data units to be retained in the target data block from the tape library buffer to the second tape library when the verification result indicates that the verification is passed.

[0266] In one embodiment, the third migration subunit 3063 can be used to:

[0267] Obtain a set of storage data units required for data migration of storage data units, wherein the set of storage data units includes at least one storage data unit to be migrated; aggregate the storage data units in the set of storage data units and the storage data units to be retained in the target data block to obtain at least one aggregated data block; migrate the aggregated data block from the tape library buffer to the second tape library.

[0268] In one embodiment, the third migration subunit 3063 may be specifically used for:

[0269] The storage data unit set is updated based on the storage data units to be retained in the target data block; the data capacity of the updated storage data unit set is calculated based on the data volume information of each storage data unit in the updated storage data unit set; when the data capacity is greater than a preset capacity threshold, the storage data units in the updated storage data unit set are aggregated.

[0270] In practice, each of the above units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units, please refer to the previous method embodiments, which will not be repeated here.

[0271] As can be seen from the above, in the data processing device of this embodiment, the acquisition unit 301 acquires a data deletion request for the tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each of the data write units has a corresponding unit state, the data block is generated by aggregating at least one storage data unit, and the data deletion request indicates the target data block to be deleted; the identification unit 302 identifies the storage data units to be deleted in the target data block according to the data deletion request, and obtains the identification result; when the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the update unit 303 updates the unit state of the write data units of the target data block; the first deletion unit 304 performs data deletion processing on each write data unit of the target data block based on the state update result of each write data unit of the target data block.

[0272] In this scheme, at least one storage data unit can be aggregated to generate a data block, and the storage data unit can be written into the tape library in units of data blocks. Specifically, the data block can be encoded into at least one write data unit and stored in the tape library. For example, each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0273] When all storage data units in the target data block to be deleted are storage data units to be deleted, this scheme updates the unit status of each write data unit in the target data block, such as updating the unit status of each write data unit in the target data block to a marked-for-deletion state. Based on the status update results of each write data unit in the target data block, for example, when the unit status of each write data unit in the target data block has been successfully updated, indicating that all tape libraries are reachable, the data is deleted from each write data unit in the target data block. In this way, through a two-stage deletion scheme of status update and status detection, it is ensured that even if some tape libraries become unavailable, each write data unit in the target data block can still be deleted successfully, thereby avoiding the generation of difficult-to-clean garbage data and improving the data management effect of tape libraries. Furthermore, because the two-stage deletion scheme in this approach sets the first stage as a lightweight state update operation rather than a heavyweight data deletion operation, it can buy time for the system to assess risks when some tape libraries become unavailable. This allows the system to reallocate computing resources to other data deletion tasks when a partial tape library becomes unavailable, rather than being blocked on the current data deletion task. This effectively reduces the probability of system blocking and improves the data processing efficiency of the tape library.

[0274] Furthermore, embodiments of this application also provide a computer device, which can be a server or terminal, etc. Figure 13 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0275] The computer device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 13 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0276] The processor 401 is the control center of the computer device, connecting various parts of the computer device through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and application programs, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0277] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0278] The computer device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0279] The computer device may also include an input unit 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0280] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the computer device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0281] A data deletion request for a tape library is obtained. The tape library stores at least one data block, which is encoded as at least one write data unit and stored in the tape library. Each write data unit has a corresponding unit state. The data block is generated by aggregating at least one storage data unit. The data deletion request indicates the target data block to be deleted. Based on the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and the identification result is obtained. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit state of the write data units of the target data block is updated. Based on the status update results of each write data unit of the target data block, data deletion processing is performed on each write data unit of the target data block.

[0282] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0283] As can be seen from the above, the computer device of this embodiment can aggregate at least one storage data unit to generate a data block, and write the storage data unit into the tape library in units of data blocks. Specifically, in this scheme, the data block can be encoded into at least one write data unit and stored in the tape library. For example, each write data unit can be stored in a different tape library to improve the disaster recovery capability and reliability of data storage.

[0284] When all storage data units in the target data block to be deleted are storage data units to be deleted, this scheme updates the unit status of each write data unit in the target data block, such as updating the unit status of each write data unit in the target data block to a marked-for-deletion state. Based on the status update results of each write data unit in the target data block, for example, when the unit status of each write data unit in the target data block has been successfully updated, indicating that all tape libraries are reachable, the data is deleted from each write data unit in the target data block. In this way, through a two-stage deletion scheme of status update and status detection, it is ensured that even if some tape libraries become unavailable, each write data unit in the target data block can still be deleted successfully, thereby avoiding the generation of difficult-to-clean garbage data and improving the data management effect of tape libraries. Furthermore, because the two-stage deletion scheme in this approach sets the first stage as a lightweight state update operation rather than a heavyweight data deletion operation, it can buy time for the system to assess risks when some tape libraries become unavailable. This allows the system to reallocate computing resources to other data deletion tasks when a partial tape library becomes unavailable, rather than being blocked on the current data deletion task. This effectively reduces the probability of system blocking and improves the data processing efficiency of the tape library.

[0285] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0286] Therefore, embodiments of this application also provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the data processing methods provided in embodiments of this application. For example, the computer program can execute the following steps:

[0287] A data deletion request for a tape library is obtained. The tape library stores at least one data block, which is encoded as at least one write data unit and stored in the tape library. Each write data unit has a corresponding unit state. The data block is generated by aggregating at least one storage data unit. The data deletion request indicates the target data block to be deleted. Based on the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and the identification result is obtained. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit state of the write data units of the target data block is updated. Based on the status update results of each write data unit of the target data block, data deletion processing is performed on each write data unit of the target data block.

[0288] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0289] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0290] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the data processing methods provided in the embodiments of this application, the beneficial effects that any of the data processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0291] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations of the data processing described above.

[0292] The data processing method, apparatus, device, and storage medium provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A data processing method, characterized in that, include: A data deletion request is obtained for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each write data unit has a corresponding unit state, the data block is generated by aggregating at least one storage data unit, and the data deletion request indicates the target data block to be deleted; wherein, the write data unit refers to the basic unit when writing data to the tape library; the storage data unit refers to the basic unit of stored data that the tape library interacts with external systems. Based on the data deletion request, the storage data units in the target data block are identified as storage data units to be deleted, and the identification result is obtained. When the identification result indicates that all storage data units in the target data block are storage data units to be deleted, the unit state of the write data units of the target data block is updated, including: updating the write data units from the current unit state to the marked-delete state; Based on the status update results of each written data unit of the target data block, data deletion processing is performed on each written data unit of the target data block, including: obtaining the status update results of each written data unit of the target data block; when the status update results indicate that all written data units of the target data block have been successfully updated, generating a data deletion task for the target data block; and performing data deletion processing on each written data unit of the target data block according to the data deletion task.

2. The data processing method according to claim 1, characterized in that, The method further includes: When the status update result indicates that the update of part of the written data unit of the target data block has failed, the update time for updating the unit status of the target data block is determined. The cell state of the write data unit of the target data block is updated at the update time; Based on the status update results of each written data unit of the target data block, data deletion processing is performed on each written data unit of the target data block.

3. The data processing method according to claim 1, characterized in that, Retrieve data deletion requests for the tape library, including: Retrieve the data deletion request for the target storage data unit; Identify the target data block containing the target storage data unit from the stored data of the tape library; Data deletion request detection is performed on each storage data unit in the target data block to obtain the detection results; When the detection result indicates that all storage data units in the target data block have corresponding data deletion requests, a data deletion request for the target data block is generated.

4. The data processing method according to claim 1, characterized in that, Retrieve data deletion requests for the tape library, including: Obtain a data migration request for a tape library, wherein the data migration request indicates a data migration of a target data block, and each storage data unit in the target data block is a storage data unit to be migrated; Determine the target tape library required for data migration of the target data block; Migrate the target data block from the tape library currently storing the target data block to the target tape library; When the data migration of the target data block is detected to be complete, a data deletion request for the target data block is generated.

5. The data processing method according to claim 4, characterized in that, Migrating the target data block from the current tape library storing the target data block to the target tape library includes: Each write data unit of the target data block is read from the tape library currently storing the target data block into the tape library buffer; Perform data verification on each of the written data units to obtain the verification results; When the verification result indicates that the verification is successful, each write data unit of the target data block is migrated from the tape library buffer to the target tape library.

6. The data processing method according to claim 1, characterized in that, The method further includes: When the identification result indicates that some storage data units in the target data block are storage data units to be retained, the first tape library currently storing the target data block and the second tape library required for data migration of the target data block are determined. The storage data units to be retained in the target data block are migrated from the first tape library to the second tape library; When migration is detected as complete, data deletion is performed on each write data unit of the target data block stored in the first tape library.

7. The data processing method according to claim 6, characterized in that, Migrating the storage data units to be retained in the target data block from the first tape library to the second tape library includes: Each write data unit of the target data block is moved from the first tape library to the tape library buffer; Perform data verification on each of the written data units to obtain the verification results; When the verification result indicates that the verification is successful, the storage data units to be retained in the target data block are migrated from the tape library buffer to the second tape library.

8. The data processing method according to claim 7, characterized in that, Migrating the storage data units to be retained in the target data block from the tape library buffer to the second tape library includes: Obtain a set of storage data units required for data migration of storage data units, wherein the set of storage data units includes at least one storage data unit to be migrated; Aggregate the storage data units in the set of storage data units and the storage data units to be retained in the target data block to obtain at least one aggregated data block; The aggregated data block is migrated from the tape library buffer to the second tape library.

9. The data processing method according to claim 8, characterized in that, Aggregating the storage data units in the set of storage data units and the storage data units to be retained in the target data block includes: The set of storage data units is updated based on the storage data units to be retained in the target data block; Calculate the data capacity of the updated storage data unit set based on the data volume information of each storage data unit in the updated storage data unit set; When the data capacity exceeds a preset capacity threshold, the storage data units in the updated storage data unit set are aggregated.

10. A data processing apparatus, characterized in that, include: An acquisition unit is used to acquire a data deletion request for a tape library, wherein the tape library stores at least one data block, the data block is encoded as at least one write data unit and stored in the tape library, each write data unit has a corresponding unit state, the data block is generated by aggregating at least one storage data unit, and the data deletion request indicates the target data block to be deleted; wherein, the write data unit refers to the basic unit when writing data to the tape library; the storage data unit refers to the basic unit of stored data that the tape library interacts with external systems. The identification unit is used to identify the storage data units to be deleted in the target data block according to the data deletion request, and obtain the identification result; An update unit is used to update the cell state of the write data unit of the target data block when the identification result indicates that all the storage data units in the target data block are storage data units to be deleted, including: updating the write data unit from the current cell state to the marked-delete state; The first deletion unit is used to perform data deletion processing on each written data unit of the target data block based on the status update result of each written data unit of the target data block. The first deletion unit includes: The first acquisition subunit is used to acquire the status update results of each written data unit of the target data block; The first generation subunit is used to generate a data deletion task for the target data block when the status update result indicates that all write data units of the target data block have been successfully updated; The first deletion subunit is used to perform data deletion processing on each written data unit of the target data block according to the data deletion task.

11. The data processing apparatus according to claim 10, characterized in that, The first deletion unit further includes: The first determining subunit is used to determine the update time for updating the unit status of the target data block when the status update result indicates that the update of part of the written data unit of the target data block has failed. A status update subunit is used to update the unit status of the write data unit of the target data block at the update time. The second deletion subunit is used to perform data deletion processing on each written data unit of the target data block based on the status update results of each written data unit of the target data block.

12. The data processing apparatus according to claim 10, characterized in that, The acquisition unit includes: The second acquisition subunit is used to acquire data deletion requests for the target storage data unit; The second determining subunit is used to determine the target data block that aggregates the target storage data unit from the stored data of the tape library; The first detection subunit is used to detect data deletion requests in each storage data unit in the target data block and obtain the detection results. The second generation subunit is used to generate a data deletion request for the target data block when the detection result indicates that all storage data units in the target data block have corresponding data deletion requests.

13. The data processing apparatus according to claim 10, characterized in that, The acquisition unit includes: The third acquisition subunit is used to acquire a data migration request for the tape library, wherein the data migration request indicates data migration of a target data block, and each storage data unit in the target data block is a storage data unit to be migrated. The third determining subunit is used to determine the target tape library required for data migration of the target data block; The first migration subunit is used to migrate the target data block from the tape library currently storing the target data block to the target tape library; The third generation subunit is used to generate a data deletion request for the target data block when it is detected that the data migration of the target data block is completed.

14. The data processing apparatus according to claim 13, characterized in that, The first migration subunit is used for: Each write data unit of the target data block is read from the tape library currently storing the target data block into the tape library buffer; data verification is performed on each write data unit to obtain a verification result; when the verification result indicates that the verification is successful, each write data unit of the target data block is migrated from the tape library buffer to the target tape library.

15. The data processing apparatus according to claim 10, characterized in that, The data processing device further includes: The determining unit is configured to determine, when the identification result indicates that some storage data units in the target data block are storage data units to be retained, a first tape library currently storing the target data block and a second tape library required for data migration of the target data block; A migration unit is used to migrate the storage data units to be retained in the target data block from the first tape library to the second tape library; The second deletion unit is used to perform data deletion processing on each write data unit of the target data block stored in the first tape library when the migration is detected to be complete.

16. The data processing apparatus according to claim 15, characterized in that, The migration unit includes: The second migration subunit is used to migrate each write data unit of the target data block from the first tape library to the tape library buffer. The verification subunit is used to verify the data in each of the written data units and obtain the verification result. The third migration subunit is used to migrate the storage data units to be retained in the target data block from the tape library buffer to the second tape library when the verification result indicates that the verification is passed.

17. The data processing apparatus according to claim 16, characterized in that, The third migration subunit is used for: Obtain a set of storage data units required for data migration of storage data units, wherein the set of storage data units includes at least one storage data unit to be migrated; aggregate the storage data units in the set of storage data units and the storage data units to be retained in the target data block to obtain at least one aggregated data block; migrate the aggregated data block from the tape library buffer to the second tape library.

18. The data processing apparatus according to claim 17, characterized in that, The third migration subunit is specifically used for: The storage data unit set is updated based on the storage data units to be retained in the target data block; the data capacity of the updated storage data unit set is calculated based on the data volume information of each storage data unit in the updated storage data unit set. When the data capacity exceeds a preset capacity threshold, the storage data units in the updated storage data unit set are aggregated.

19. A computer device, characterized in that, It includes a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to perform the data processing method according to any one of claims 1 to 9.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to perform the data processing method according to any one of claims 1 to 9.

21. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the data processing method according to any one of claims 1 to 9.

Images