File deletion methods, systems, devices, processors, and electronic devices

Abstract

This application discloses a file deletion method, system, apparatus, processor, and electronic device. The method includes: a master metadata system receiving at least one deletion request from a client; the master metadata system marking the target directory in the recycle bin as deleted and determining whether the recycle bin's state is a no-task state; if the recycle bin's state is no-task, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, updating the recycle bin's state to a task processing state, and recording the cursor and operation identifier of the deletion operation; if the master metadata system detects that the cursors of all threads indicate that all files in the target directory have been deleted, the recycle bin's state is updated to a no-task state. This application solves the problem of low efficiency when deleting files from the recycle bin in related technologies.

Description

Technical Field

[0001] This application relates to the field of file deletion, and more specifically, to a file deletion method, system, apparatus, processor, and electronic device. Background Technology

[0002] In related technologies, the recycle bin performs deletion and emptying actions synchronously. For example, when a file (directory) is selected for deletion on the front-end interface or when the recycle bin is emptied, the deletion operation can only be completed after all files and subdirectories under that file (directory) have been deleted, or all files and subdirectories in the recycle bin have been deleted. During the deletion operation, the front-end interface will be blocked on that operation interface and cannot perform other operations. If there are many files in the directory to be deleted or the recycle bin to be emptied, the front-end interface will be blocked for a long time.

[0003] Meanwhile, during a deletion operation, the client typically sends a `readdir` (directory traversal) request to the `mds` (metadata system). The `mds` then returns a partial traversal result (the directory may be very large, so only a portion can be returned) to the client. The client then sends deletion requests for each traversed file, and the `mds` processes these requests. Therefore, multiple network transmissions and `mds` processing are required. The more files in the directory to be deleted or the more files in the recycle bin, the lower the efficiency of this operation becomes.

[0004] Additionally, if the main MDS process of the distributed file system unexpectedly exits during the deletion of a directory or emptying of the recycle bin, the main MDS will switch to another MDS, and the deletion of the directory and emptying of the recycle bin will need to be re-executed on the new MDS. Alternatively, if the client process that initiated the request unexpectedly exits, the deletion of the directory and emptying of the recycle bin will also fail to continue. When viewed from other front-end interfaces, the directory to be deleted still exists, and the recycle bin to be emptied has not yet been emptied.

[0005] There is currently no effective solution to the problem of low efficiency in deleting files from the recycle bin in related technologies. Summary of the Invention

[0006] The main objective of this application is to provide a file deletion method, system, device, processor, and electronic device to solve the problem of low efficiency when deleting files from the recycle bin in related technologies.

[0007] To achieve the above objectives, according to one aspect of this application, a file deletion method is provided. The method includes: a master metadata system receiving at least one deletion request from a client, wherein the deletion request indicates the deletion of a target directory within the recycle bin; the master metadata system marking the target directory within the recycle bin as deleted, and determining whether the recycle bin's state is a no-task state, wherein a no-task state means there are no deletion operations in the recycle bin; when the recycle bin's state is a no-task state, the master metadata system controls a target thread to execute the deletion operation corresponding to each deletion request, updating the recycle bin's state to a task processing state, and recording the cursor and operation identifier of the deletion operation, wherein the cursor is used to record the number of files and bytes currently deleted, and the operation identifier is an index used by the client to query the deletion progress of the deletion operation; when the master metadata system detects that the cursors of all threads indicate that all files in the target directory have been deleted, the recycle bin's state is updated to a no-task state.

[0008] Optionally, before the primary metadata system marks the target directory as deleted, the method further includes: the primary metadata system sending a deletion request to multiple backup metadata systems and controlling the multiple backup metadata systems to store the deletion request; after the primary metadata system detects that the multiple backup metadata systems have stored the deletion request, sending the file information of the target directory to the client, wherein the file information is used to indicate the number of files to be deleted in the target directory, the number of bytes to be deleted, and the operation identifier corresponding to the deletion request.

[0009] Optionally, during the process of the primary metadata system controlling the target thread to execute the deletion operation corresponding to each deletion request, the method further includes: if the primary metadata system malfunctions, switching the primary metadata system to any one of the multiple backup metadata systems to obtain the switched primary metadata system; the switched primary metadata system reads the cursors of each deletion operation and continues to execute the deletion operation according to the number of currently deleted files, the number of bytes, and the position of the file being deleted recorded in the cursor, where the file position refers to the position of the file being deleted in the target directory when the primary metadata system malfunctions.

[0010] Optionally, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, and records the cursor of the deletion operation, including: traversing all files in the target directory to obtain a set of files to be deleted; deleting files in the set of files to be deleted in batches according to a preset parallel quantity, wherein the preset parallel quantity is the parallel quantity of a deletion operation; and updating the cursor in real time according to the changes in the number of deleted files and the number of deleted bytes in the set of files to be deleted.

[0011] Optionally, after determining whether the recycle bin is in a no-task state, the method further includes: if the recycle bin is in a delete directory state, the main metadata system determines whether there is an idle thread; if there is an idle thread, it controls any one of the idle threads to execute the deletion operation corresponding to the deletion request; if the recycle bin is in a cleared recycle bin state, the main metadata system refuses to execute the deletion operation corresponding to the deletion request.

[0012] Optionally, after recording the cursor of the deletion operation, the method further includes: the master metadata system receiving the client's deletion progress query request, parsing the deletion progress query request, and obtaining the operation identifier of the target deletion operation; determining the deletion operation that the client needs to query based on the operation identifier of the target deletion operation, and obtaining the target deletion operation; reading the target cursor of the target deletion operation, and displaying the current number of deleted files and the current number of deleted bytes of the target deletion operation to the client.

[0013] Optionally, the recycle bin is associated with a first audit log, which contains records of files being moved into and out of the recycle bin. The first audit log is generated according to a preset period. After the master metadata system receives at least one deletion request from the client, the method further includes: determining whether the space occupied by the first audit log is greater than or equal to a space threshold; if the space occupied by the first audit log is greater than or equal to the space threshold, establishing a second audit log and recording the deletion operation corresponding to the deletion request in the second audit log; if the space occupied by the first audit log is less than the space threshold, recording the deletion operation corresponding to the deletion request in the first audit log.

[0014] Optionally, both the deletion operation and the audit log are set with levels. Before recording the deletion operation to the audit log, the method further includes: determining whether the level of the deletion operation is less than or equal to the level of the audit log; if the level of the deletion operation is less than or equal to the level of the audit log, performing the step of recording the deletion operation to the audit log.

[0015] According to another aspect of this application, a file deletion system is provided, the system comprising: a file system for storing files in a target directory; a recycle bin for storing files in the target directory moved from the file system to the recycle bin; a client for issuing deletion requests, wherein the deletion request instructs the deletion of files in the target directory in the recycle bin; and a metadata cluster including a primary metadata system and a backup metadata system, wherein the primary metadata system receives deletion requests and controls a target thread to perform an operation to delete the target directory from the recycle bin based on the deletion request, and the backup metadata system stores deletion requests sent by the primary metadata system and switches the backup metadata system to the primary metadata system when the primary metadata system malfunctions.

[0016] To achieve the above objectives, according to another aspect of this application, a file deletion apparatus is provided. The apparatus includes: a receiving unit, configured to receive at least one deletion request from a client via a master metadata system, wherein the deletion request indicates the deletion of a target directory within the recycle bin; a judging unit, configured to mark the target directory within the recycle bin as deleted via the master metadata system, and judge whether the recycle bin's state is a no-task state, wherein a no-task state means that no deletion operation exists in the recycle bin; an execution unit, configured to, when the recycle bin's state is a no-task state, control a target thread via the master metadata system to execute the deletion operation corresponding to each deletion request, update the recycle bin's state to a task processing state, and record the cursor and operation identifier of the deletion operation, wherein the cursor records the number of files and bytes currently deleted, and the operation identifier is an index for the client to query the deletion progress of the deletion operation; and an updating unit, configured to update the recycle bin's state to a no-task state when the master metadata system detects that the cursors of all threads indicate that all files in the target directory have been deleted.

[0017] This application employs the following steps: the master metadata system receives at least one deletion request from a client, wherein the deletion request indicates the deletion of a target directory within the recycle bin; the master metadata system marks the target directory within the recycle bin as deleted and determines whether the recycle bin's state is a no-task state, wherein a no-task state means that there are no deletion operations in the recycle bin; when the recycle bin's state is a no-task state, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, updates the recycle bin's state to a task processing state, and records the cursor and operation identifier of the deletion operation, wherein the cursor is used to record the number of files and bytes currently deleted, and the operation identifier is an index for the client to query the deletion progress of the deletion operation; when the master metadata system detects that the cursors of all threads indicate that all files in the target directory have been deleted, it updates the recycle bin's state to a no-task state, thus solving the problem of low efficiency when deleting files from the recycle bin in related technologies. After receiving a deletion request, the target directory to be deleted is marked as deleted. Then, the control thread executes the operation corresponding to the deletion request. At the same time, the deletion operation record cursor and operation identifier are used to query the deletion progress. This avoids the need to show every file in the target directory to the client when performing the deletion operation, thereby improving the efficiency of deleting files from the recycle bin. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0019] Figure 1This is a flowchart of a file deletion method provided according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the metadata system provided according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the audit log workflow provided according to an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a file deletion system provided according to an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of a file deletion device provided according to an embodiment of this application. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0028] The present invention will now be described in conjunction with preferred implementation steps. Figure 1This is a flowchart of a file deletion method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0029] In step S101, the master metadata system receives at least one deletion request from the client, wherein the deletion request is used to instruct the deletion of the target directory within the recycle bin.

[0030] Specifically, the metadata system (MDS), also known as the master metadata system, is a system used to manage, store, and distribute metadata information in a distributed file system. A client is the endpoint through which a user sends commands to publish files within the file system; a deletion request is a user's instruction to delete a directory from the recycle bin. Clients can send requests to delete directories in the recycle bin or empty the recycle bin directly to the master MDS.

[0031] in, Figure 2 This is a schematic diagram of the metadata system provided according to an embodiment of this application, such as... Figure 2 As shown, the metadata system includes a network layer, a service layer, a distributed processing layer, and a local persistence layer.

[0032] Transport represents the network layer, responsible for receiving and sending network packets. MDSService represents the service layer, used to forward received and parsed network packets to the underlying interface for processing; it is also responsible for submitting the network packet requests processed by the underlying interface back to the network layer for the client. MetaGroup represents the distributed processing layer, responsible for performing a series of legality checks on the deletion requests forwarded from MDSService (duplicate operation checks, operator permission checks, operation directory permission checks, and disk space full checks, etc.) and forming a transaction (a set of files that need to be modified) to forward to other slave MDSs, and after waiting for all slave MDSs to return, it submits the request back to the MDSService layer.

[0033] MDSStore represents the local persistence layer, responsible for persisting transactions formed by the MetaGroup layer to the local disk (database). The trash namespace represents the recycle bin space, used to store all directories and files moved into the recycle bin. Directories and files in this space can be recovered or permanently deleted later. The gc namespace represents the garbage collection space, organizing all records within this space according to certain rules. The files corresponding to the records in this space have been deleted and cannot be recovered; the purpose of these records is to release the disk space occupied by the data portion of deleted files later.

[0034] In step S102, the master metadata system marks the target directory in the recycle bin as deleted and determines whether the recycle bin is in a no-task state, where no-task state means that there is no deletion operation in the recycle bin.

[0035] Specifically, the target directory is the directory stored in the recycle bin. The deletion request instructs the target directory to be deleted from the recycle bin. A no-task state means that no operation is currently being performed on the recycle bin. Operations performed on the recycle bin include deleting directory A, emptying the recycle bin, etc.

[0036] It's important to note that when a deletion request instructs the recycle bin to be emptied, the system checks if the recycle bin directory is empty. If it's not empty, the `current` directory within the recycle bin directory is renamed to a timestamp directory `~archive.xxx`. The `current` directory contains all files in the recycle bin except for the `archive` directory. `~archive.xxx` and other existing timestamp directories within the recycle bin are marked as deleted, so this directory and its subdirectories and files will not be traversed or displayed during subsequent iterations.

[0037] In step S103, when the recycle bin is in a no-task state, the main metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, updates the recycle bin's state to a task processing state, and records the cursor and operation identifier of the deletion operation. The cursor is used to record the number of files and bytes currently deleted, and the operation identifier is the index for the client to query the deletion progress of the deletion operation.

[0038] Specifically, the target thread refers to the thread controlled by the main metadata system used to perform deletion operations. The task processing status indicates that there are currently tasks executing in the recycle bin, such as deleting the target directory or emptying the recycle bin. The operation identifier is the version number of the deletion operation; the operation identifier allows you to find the corresponding deletion operation. The target thread traverses the target directory and performs deletion operations in a batch file deletion manner. The main metadata system records this deletion operation and stores a cursor, so even if the system switches from one metadata system to the main metadata system during traversal and deletion, the new metadata system can continue the traversal and deletion process from the cursor, thus avoiding the performance degradation caused by restarting the traversal and scanning for already deleted files (database key-value pairs).

[0039] Step S104: When the main metadata system detects that all files in the target directory have been deleted as indicated by the cursors of all threads, the status of the recycle bin is updated to a no-task state.

[0040] Specifically, when the main MDS completes all deletion requests under the recycle bin, it changes the recycle bin's status to "no task". When the next deletion request needs to be processed, it can be executed directly when the recycle bin is in a "no task" state, avoiding waiting for other deletion requests being executed in the recycle bin to finish before executing the next deletion request.

[0041] The file deletion method provided in this application embodiment receives at least one deletion request from a client through a master metadata system. The deletion request indicates the deletion of a target directory within the recycle bin. The master metadata system marks the target directory within the recycle bin as deleted and determines whether the recycle bin is in a no-task state, where no-task means there are no deletion operations in the recycle bin. When the recycle bin is in a no-task state, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, updates the recycle bin's state to a task processing state, and records the cursor and operation identifier of the deletion operation. The cursor records the number of files and bytes currently deleted, and the operation identifier is the index for the client to query the deletion progress. When the master metadata system detects that all thread cursors indicate that all files in the target directory have been deleted, it updates the recycle bin's state to a no-task state, thus solving the problem of low efficiency when deleting files from the recycle bin in related technologies. After receiving a deletion request, the target directory to be deleted is marked as deleted. Then, the control thread executes the operation corresponding to the deletion request. At the same time, the deletion operation record cursor and operation identifier are used to query the deletion progress. This avoids the need to show every file in the target directory to the client when performing the deletion operation, thereby improving the efficiency of deleting files from the recycle bin.

[0042] To avoid the need for reprocessing of ongoing deletion requests in the event of a failure in the primary metadata system, optionally, in the file deletion method provided in this application embodiment, before the primary metadata system marks the target directory as deleted, the method further includes: the primary metadata system sending deletion requests to multiple backup metadata systems and controlling the multiple backup metadata systems to store the deletion requests; after the primary metadata system detects that the multiple backup metadata systems have stored the deletion requests, sending the file information of the target directory to the client, wherein the file information is used to indicate the number of files to be deleted in the target directory, the number of bytes to be deleted, and the operation identifier corresponding to the deletion request.

[0043] Specifically, the backup metadata system, also known as the slave MDS, is responsible for taking over the primary MDS's tasks within the file system when the primary MDS fails. The slave MDS stores the specific content information within the target directory, including the number of files and their bytes. The primary MDS distributes and persists deletion requests across the slave MDSs, meaning each slave MDS stores the deletion request. This ensures that if the primary MDS fails and another slave MDS needs to be switched to primary, the deletion request can still be processed by the new primary MDS. Once all slave MDSs have completed the persistence process, the primary MDS returns file information to the client, indicating the number of files and their bytes to be deleted in the target directory. This process of persisting deletion requests on slave MDSs and returning file information to the client is highly efficient, without any blocking on the front-end interface. By storing deletion requests in the backup metadata system, the loss of deletion requests after switching from a slave to a primary metadata system is avoided, preventing the need to re-initiate deletion requests.

[0044] When the primary metadata system fails and the backup metadata system is switched to the primary metadata system, the new primary metadata system can restore the deletion operation before the switch by reading the cursor of the deletion operation. Optionally, in the file deletion method provided in this application embodiment, during the process of the primary metadata system controlling the target thread to execute the deletion operation corresponding to each deletion request, the method further includes: if the primary metadata system is abnormal, switching the primary metadata system to any one of the multiple backup metadata systems to obtain the switched primary metadata system; the switched primary metadata system reads the cursor of each deletion operation and continues to execute the deletion operation according to the number of currently deleted files, the number of bytes, and the position of the file being deleted recorded in the cursor, wherein the file position refers to the position of the file being deleted in the target directory when the primary metadata system is abnormal.

[0045] Specifically, an anomaly in the primary metadata system refers to the unexpected exit of a process executing a deletion operation within the primary metadata system. In this case, it is necessary to switch any one of the multiple backup metadata systems to the primary metadata system. The new primary metadata system then continues the deletion operation that was unexpectedly terminated. By reading the cursor of the deletion operation, the execution progress of the process before the switch can be obtained, i.e., the files that have been deleted, the bytes that have been deleted, and the position of the file being deleted. The deletion operation can then continue from the remaining files. By reading the cursor after switching the primary metadata system and continuing the unfinished deletion operation, the problem of having to re-execute the process of a deletion operation that has already been performed is avoided after the primary metadata system switch.

[0046] Optionally, in the file deletion method provided in this application embodiment, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, and records the cursor of the deletion operation, including: traversing all files in the target directory to obtain a set of files to be deleted; deleting files in the set of files to be deleted in batches according to a preset parallel quantity, wherein the preset parallel quantity is the parallel quantity of a deletion operation; and updating the cursor in real time according to the changes in the number of deleted files and the number of deleted bytes in the set of files to be deleted.

[0047] Specifically, the set of files to be deleted is the collection of all files stored in the target directory, and the preset parallelism is the number of files that the target thread can delete simultaneously. When the target thread performs a deletion operation, it first traverses all files stored in the target directory to obtain the set of files to be deleted. It then sets the preset parallelism according to the number of files the target thread can delete simultaneously, and deletes the files in the set of files to be deleted in batches according to the preset parallelism. Since the deletion process is time-consuming, to avoid affecting other operations on the front-end interface, the set of files to be deleted after the target thread has traversed it is not sent to the client. The client can query the progress of the deletion operation in real time through the operation identifier in the file information. The progress of the deletion operation is recorded using a cursor. Each time the target thread executes the deletion of a batch of files, the cursor is updated once to show the number of files deleted and the number of bytes deleted. By updating the deletion progress of the deletion operation in real time through the cursor, and without returning the traversed files to the client, the time spent communicating and interacting between the client and the metadata system is saved, thus improving the efficiency of file deletion.

[0048] Since the thread executing the deletion operation in the recycle bin is no longer displayed to the client, it is necessary to determine the state of the recycle bin before executing a new deletion request. Optionally, in the file deletion method provided in this application embodiment, after determining whether the state of the recycle bin is a no-task state, the method further includes: if the state of the recycle bin is a delete directory state, the main metadata system determines whether there is an idle thread; if there is an idle thread, it controls any one of the idle threads to execute the deletion operation corresponding to the deletion request; if the state of the recycle bin is an empty recycle bin state, the main metadata system refuses to execute the deletion operation corresponding to the deletion request.

[0049] Specifically, the recycle bin's working states include no task, deleted directory, empty recycle bin, and deleted recycle bin. When the recycle bin is in a "no task" state, it means no deletion operations are being performed, and all threads in the main metadata system are idle. Upon receiving a deletion request, the main metadata system can directly control the target thread to execute the corresponding deletion operation. When the recycle bin is in a "deleted directory" state, it means one or more deletion operations are being performed. It's necessary to first determine if there are any idle threads in the main metadata system. If there are, the deletion operation can be executed; otherwise, it must wait for the thread currently performing a deletion operation in the recycle bin to finish before executing the deletion operation. When the recycle bin is in a "empty recycle bin" or "deleted recycle bin" state, since all files in the recycle bin are in the process of deletion, the main metadata system refuses to execute the deletion request to avoid thread conflicts. By checking the recycle bin's state before executing a deletion operation, thread conflicts in the main metadata system that could lead to malfunctions are avoided.

[0050] When the deletion operation corresponding to the deletion request is executed, since the master metadata system does not return the traversed set of files to be deleted to the client, when the client needs to know the deletion process, it can query the deletion process through the operation identifier. Optionally, in the file deletion method provided in this application embodiment, after recording the cursor of the deletion operation, the method further includes: the master metadata system receiving the client's deletion progress query request and parsing the deletion progress query request to obtain the operation identifier of the target deletion operation; determining the deletion operation that the client needs to query based on the operation identifier of the target deletion operation to obtain the target deletion operation; reading the target cursor of the target deletion operation and displaying the current number of deleted files and the current number of deleted bytes of the target deletion operation to the client.

[0051] Specifically, a deletion progress query request refers to a request sent by the client to the main metadata system when a deletion operation is in progress, since the main metadata system does not return the traversed set of files to be deleted to the client. Each deletion operation corresponds to a unique operation identifier, i.e., version (the version number of the deletion operation). By parsing the deletion progress query request, the operation identifier of the target deletion operation is obtained. Based on the operation identifier, the target deletion operation that the client needs to query is determined from the multiple deletion operations currently being executed in the recycle bin. The cursor of the target deletion operation is read to obtain the deletion progress of the target deletion operation that the client needs to query, i.e., the number of files currently deleted and the number of bytes deleted. By providing the deletion progress query function, instead of returning the traversed set of files to be deleted to the client, the communication process between the main metadata system and the client is streamlined, improving file deletion efficiency.

[0052] After deleting files from the recycle bin, the recycle bin also provides a logging function. Optionally, in the file deletion method provided in this application embodiment, the recycle bin is associated with a first audit log. The first audit log contains records of files being moved into and out of the recycle bin. The first audit log is generated according to a preset period. After the master metadata system receives at least one deletion request from the client, the method further includes: determining whether the space occupied by the first audit log is greater than or equal to a space threshold; if the space occupied by the first audit log is greater than or equal to the space threshold, establishing a second audit log and recording the deletion operation corresponding to the deletion request to the second audit log; if the space occupied by the first audit log is less than the space threshold, recording the deletion operation corresponding to the deletion request to the first audit log.

[0053] Specifically, the first audit log is a periodically updated log recording various operations performed on the recycle bin. The preset cycle is one day. To ensure readability, the audit log is segmented according to a specific file size. When the space occupied by the first audit log exceeds a set threshold for the day, it is closed, and a new second audit log is created to record the recycle bin's operations, such as file deletion. This prevents the log file from becoming too large and difficult to read. By providing the recycle bin's log records, users can easily query the various operations performed on the recycle bin.

[0054] It should be noted that the recycle bin is generally the last layer of protection against accidental deletion of files. Therefore, some applications need to strictly audit the process of files entering the recycle bin and being restored or deleted from the recycle bin. Figure 3 This is a schematic diagram of the audit log workflow provided in the embodiments of this application, such as... Figure 3 As shown, to improve audit log recording efficiency, the master metadata system pre-allocates eight different log pools (log resource pools) to reduce contention during concurrent resource requests. Logs are allocated from these pools to record various operations of the recycle bin. Completed logs are inserted into a linked list, and a dedicated thread scans the list and sends them in batches to the Rados (object storage) cluster, where they are written to specific files. This audit log workflow provides a more intuitive and readable view of the audit log file information and content.

[0055] When the audit log records a deletion operation in the recycle bin, it needs to first determine whether the level of the deletion operation is less than or equal to the level set in the audit log. Optionally, in the file deletion method provided in this application embodiment, both the deletion operation and the audit log are set with levels. Before recording the deletion operation to the audit log, the method further includes: determining whether the level of the deletion operation is less than or equal to the level of the audit log; if the level of the deletion operation is less than or equal to the level of the audit log, performing the step of recording the deletion operation to the audit log.

[0056] Specifically, the level, or priority, is the level set for various types of operations in the recycle bin, including low, medium, and high. The audit log will set the highest level for recording operations in the recycle bin, meaning that only operations with a priority less than or equal to the level set in the audit log will be recorded in the audit log.

[0057] For example, since the operation of moving a file to the recycle bin is on the path of the file system's deletion operation, to avoid affecting the efficiency of file deletion, the priority of this type of audit log operation is set to high, and the audit log level is set to medium. High is higher than medium, therefore, the operation of moving a file to the recycle bin will not be recorded in the log. The operation of emptying the recycle bin is low, which is lower than medium, therefore, the operation of emptying the recycle bin will be recorded in the audit log. The operation of deleting a file in the recycle bin is medium, which is equal to the level set in the audit log, therefore, this operation will also be recorded in the audit log. Table 1 shows the various recycle bin operations and priorities provided according to the embodiments of this application.

[0058] Table 1

[0059]

[0060]

[0061] According to another aspect of this application, a file deletion system is provided. Figure 4 This is a schematic diagram of the structure of a file deletion system provided according to an embodiment of this application, such as... Figure 4 As shown, the system includes:

[0062] File system 41 is used to store files in the target directory;

[0063] Specifically, the file system 41 is a system that stores all the files that the client 43 needs to process. When the client 43 needs to delete a file or directory, it first moves the file or directory into the recycle bin 42, and then deletes the file or directory from the recycle bin 42.

[0064] Recycle bin 42 is used to store files moved from file system 41 to the target directory of recycle bin 42;

[0065] Specifically, the recycle bin 42 is used to store files deleted from the file system 41. Files in the recycle bin 42 can be deleted or restored to the file system 41.

[0066] Client 43 is used to issue a deletion request, wherein the deletion request instructs to delete files in the target directory in the recycle bin 42;

[0067] Specifically, client 43 refers to the front end of user operation. After the user issues a deletion request, the deletion request instructs metadata cluster 44 to execute the instruction in the deletion request to delete the file in the target directory from recycle bin 42.

[0068] Metadata cluster 44 includes a primary metadata system and a backup metadata system. The primary metadata system is used to receive deletion requests and control the target thread to perform the operation of deleting the target directory from the recycle bin 42 based on the deletion request. The backup metadata system is used to store the deletion requests sent by the primary metadata system and to switch the backup metadata system to the primary metadata system when the primary metadata system fails.

[0069] Specifically, the metadata cluster 44 contains multiple metadata systems, with only one primary metadata system at any given time. The other metadata systems are backup systems. If the primary metadata system fails, any backup metadata system can switch to become the primary system and continue executing any unfinished operations from the previous primary system. The primary metadata system is responsible for controlling the threads that execute the deletion operations corresponding to the deletion requests.

[0070] The file deletion system provided in this application embodiment involves a client 43 issuing a deletion request. The primary metadata system in the metadata cluster 44 executes the deletion request by first moving the target directory from the file system 41 to the recycle bin 42, and then deleting the target directory from the recycle bin 42. This reduces the communication interaction between the client 43 and the primary metadata system, saves the time spent deleting files, and improves file recycling efficiency.

[0071] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0072] This application also provides a file deletion device. It should be noted that the file deletion device of this application can be used to execute the file deletion method provided in this application. The file deletion device provided in this application is described below.

[0073] Figure 5This is a schematic diagram of a file deletion device provided according to an embodiment of this application. For example... Figure 5 As shown, the device includes:

[0074] The receiving unit 10 is configured to receive at least one deletion request from a client through the master metadata system, wherein the deletion request is used to instruct the deletion of a target directory within the recycle bin;

[0075] The judgment unit 20 is used to mark the target directory in the recycle bin as deleted through the master metadata system, and to determine whether the status of the recycle bin is a no-task state, where no-task state means that there is no deletion operation in the recycle bin;

[0076] The execution unit 30 is used to control the target thread to execute the deletion operation corresponding to each deletion request when the recycle bin is in a no-task state. It updates the recycle bin's state to a task processing state and records the cursor and operation identifier of the deletion operation. The cursor is used to record the number of files and bytes that have been deleted. The operation identifier is the index for the client to query the deletion progress of the deletion operation.

[0077] Update unit 40 is used to update the status of the recycle bin to a no-task state when the main metadata system detects that all files in the target directory have been deleted as indicated by the cursors of all threads.

[0078] The file deletion device provided in this application embodiment receives at least one deletion request from a client through a receiving unit 10 via a master metadata system. The deletion request indicates the deletion of a target directory within the recycle bin. A judging unit 20 marks the target directory within the recycle bin as deleted through the master metadata system and determines whether the recycle bin is in a no-task state, where no-task means there is no deletion operation in the recycle bin. An execution unit 30, when the recycle bin is in a no-task state, controls the target thread through the master metadata system to execute the deletion operation corresponding to each deletion request, updates the recycle bin's state to a task processing state, and records the cursor and operation identifier of the deletion operation. The cursor is used to record currently deleted... The number of files and bytes, and the operation identifier serve as an index for the client to query the deletion progress of the deletion operation; the update unit 40, when the main metadata system detects that all files in the target directory have been deleted as indicated by the cursors of all threads, updates the state of the recycle bin to a no-task state, which solves the problem of low efficiency when deleting files from the recycle bin in related technologies. By receiving the deletion request, the target directory to be deleted is directly marked as deleted, and then the thread is controlled to execute the operation corresponding to the deletion request. At the same time, the deletion operation record cursor and operation identifier are used to query the deletion progress, avoiding the need to show each file in the target directory to the client when performing the deletion operation, thereby improving the efficiency of deleting files from the recycle bin.

[0079] Optionally, in the file deletion device provided in the embodiments of this application, the device further includes: a first sending unit, configured to send a deletion request to multiple backup metadata systems through a primary metadata system, and control the multiple backup metadata systems to store the deletion request; and a second sending unit, configured to send file information of the target directory to the client after the primary metadata system detects that multiple backup metadata systems have stored the deletion request, wherein the file information is used to indicate the number of files to be deleted in the target directory, the number of bytes to be deleted, and the operation identifier corresponding to the deletion request.

[0080] Optionally, in the file deletion device provided in this application embodiment, the device further includes: a switching unit, used to switch the main metadata system to any one of the multiple backup metadata systems if the main metadata system malfunctions, to obtain the switched main metadata system; and a first reading unit, used to read the cursors of each deletion operation through the switched main metadata system, and continue to execute the deletion operation according to the number of currently deleted files, the number of bytes, and the position of the file being deleted recorded in the cursor, wherein the file position refers to the position of the file being deleted in the target directory when the main metadata system malfunctions.

[0081] Optionally, in the file deletion device provided in this application embodiment, the execution unit 30 includes: a traversal module, used to traverse all files in the target directory to obtain a set of files to be deleted; a deletion module, used to delete files in the set of files to be deleted in batches according to a preset parallel quantity, wherein the preset parallel quantity is the parallel quantity of a deletion operation; and an update module, used to update the cursor in real time according to the changes in the number of deleted files and the changes in the number of deleted bytes in the set of files to be deleted.

[0082] Optionally, in the file deletion device provided in this application embodiment, the device further includes: a thread judgment unit, used to determine whether there is an idle thread when the state of the recycle bin is the state of the deleted directory; a control unit, used to control any one of the idle threads to execute the deletion operation corresponding to the deletion request when there is an idle thread; and a rejection execution unit, used to refuse to execute the deletion operation corresponding to the deletion request when the state of the recycle bin is the state of the emptied recycle bin.

[0083] Optionally, in the file deletion device provided in this application embodiment, the device further includes: a parsing unit, used to receive a deletion progress query request from a client through a master metadata system, and parse the deletion progress query request to obtain an operation identifier of the target deletion operation; a determining unit, used to determine the deletion operation that the client needs to query based on the operation identifier of the target deletion operation, and obtain the target deletion operation; and a second reading unit, used to read the target cursor of the target deletion operation and display to the client the current number of deleted files and the current number of deleted bytes of the target deletion operation.

[0084] Optionally, in the file deletion device provided in this application embodiment, the recycle bin is associated with a first audit log, which contains records of files being moved into and out of the recycle bin. The first audit log is generated according to a preset period. The device further includes: a space occupancy judgment unit, used to determine whether the space occupied by the first audit log is greater than or equal to a space threshold; an establishment unit, used to establish a second audit log when the space occupied by the first audit log is greater than or equal to the space threshold, and record the deletion operation corresponding to the deletion request to the second audit log; and a recording unit, used to record the deletion operation corresponding to the deletion request to the first audit log when the space occupied by the first audit log is less than the space threshold.

[0085] Optionally, in the file deletion device provided in this application embodiment, both the deletion operation and the audit log are set with levels. The device further includes: a level judgment unit, used to determine whether the level of the deletion operation is less than or equal to the level of the audit log; and a recording execution unit, used to execute the step of recording the deletion operation to the audit log when the level of the deletion operation is less than or equal to the level of the audit log.

[0086] The file deletion device includes a processor and a memory. The receiving unit 10, the judging unit 20, the execution unit 30, and the updating unit 40 are all stored in the memory as program units. The processor executes the program units stored in the memory to achieve the corresponding functions.

[0087] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and kernel parameters can be adjusted to improve the efficiency of deleting files from the recycle bin.

[0088] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0089] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements a file deletion method.

[0090] This invention provides a processor for running a program, wherein the program executes a file deletion method during runtime.

[0091] This invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: a file deletion method. The device described herein can be a server, PC, PAD, mobile phone, etc.

[0092] This application also provides a computer program product that, when executed on a data processing device, is suitable for executing a program with the following initialization steps: a file deletion method.

[0093] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0094] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0095] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0096] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0097] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0098] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0099] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0100] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0101] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A file deletion method characterized by, include: The master metadata system receives at least one deletion request from a client, wherein the deletion request is used to instruct the deletion of a target directory within the recycle bin; The master metadata system marks the target directory in the recycle bin as deleted and determines whether the recycle bin is in a no-task state, where no-task state means that there is no deletion operation in the recycle bin; When the recycle bin is in a no-task state, the master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, updates the recycle bin's state to a task processing state, and records the cursor and operation identifier of the deletion operation. The cursor is used to record the number of files and bytes currently deleted, and the position of the file being deleted. The operation identifier is the index for the client to query the deletion progress of the deletion operation. Once the master metadata system detects that all files in the target directory have been deleted as indicated by the cursors of all threads, the status of the recycle bin will be updated to a no-task state.

2. The method of claim 1, wherein, The method further includes the following steps before the master metadata system marks the target directory as deleted: The primary metadata system sends the deletion request to multiple backup metadata systems and controls the multiple backup metadata systems to store the deletion request; After the primary metadata system detects that the multiple backup metadata systems have stored the deletion request, it sends the file information of the target directory to the client. The file information is used to indicate the number of files to be deleted in the target directory, the number of bytes to be deleted, and the operation identifier corresponding to the deletion request.

3. The method of claim 2, wherein, During the process of the master metadata system controlling the target thread to execute the deletion operation corresponding to each deletion request, the method further includes: If the primary metadata system malfunctions, it will be switched to any one of the multiple backup metadata systems to obtain the switched primary metadata system. The switched master metadata system reads the cursors of each deletion operation and continues to execute the deletion operation according to the number of files deleted, the number of bytes, and the position of the file being deleted recorded in the cursor. The file position refers to the position of the file being deleted in the target directory when the master metadata system malfunctions.

4. The method of claim 1, wherein, The master metadata system controls the target thread to execute the deletion operation corresponding to each deletion request, and records the cursor of the deletion operation, including: Traverse all files in the target directory to obtain the set of files to be deleted; Files in the set of files to be deleted are deleted in batches according to a preset parallel quantity, wherein the preset parallel quantity is the parallel quantity of a deletion operation; The cursor is updated in real time based on changes in the number of deleted files and the number of deleted bytes in the set of files to be deleted.

5. The method of claim 1, wherein, After determining whether the recycle bin is in a no-task state, the method further includes: If the state of the recycle bin is not the no-task state, determine whether the state of the recycle bin is the state of deleting a directory, or determine whether the state of the recycle bin is the state of emptying the recycle bin; When the recycle bin is in the state of a deleted directory, the master metadata system determines whether there is an idle thread; If an idle thread exists, control any one of the idle threads to execute the deletion operation corresponding to the deletion request; If the recycle bin is in an empty state, the master metadata system refuses to execute the deletion operation corresponding to the deletion request.

6. The method of claim 1, wherein, After recording the cursor of the deletion operation, the method further includes: The master metadata system receives the deletion progress query request from the client, parses the deletion progress query request, and obtains the operation identifier of the target deletion operation; The deletion operation that the client needs to query is determined based on the operation identifier of the target deletion operation, and the target deletion operation is obtained; Read the target cursor of the target deletion operation and display the current number of files deleted and the current number of bytes deleted to the client.

7. The method of claim 1, wherein, The recycle bin is associated with a first audit log, which contains records of files being moved into and out of the recycle bin. The first audit log is generated at a preset period. After the master metadata system receives at least one deletion request from a client, the method further includes: Determine whether the space occupied by the first audit log is greater than or equal to the space threshold; If the space occupied by the first audit log is greater than or equal to the space threshold, a second audit log is created, and the deletion operation corresponding to the deletion request is recorded in the second audit log; If the space occupied by the first audit log is less than the space threshold, the deletion operation corresponding to the deletion request is recorded in the first audit log.

8. The method of claim 7, wherein, Both the deletion operation and the audit log are configured with levels. Before recording the deletion operation to the audit log, the method further includes: Determine whether the level of the deletion operation is less than or equal to the level of the audit log; If the level of the deletion operation is less than or equal to the level of the audit log, the step of recording the deletion operation to the audit log is performed.

9. A file erasing system applied to the file erasing method as claimed in claim 1, characterized by, include: The file system is used to store files in the target directory; A recycle bin is used to store files moved from the file system to the target directory. A client is configured to issue a deletion request, wherein the deletion request indicates the deletion of files in a target directory in the recycle bin; The metadata cluster includes a primary metadata system and a backup metadata system. The primary metadata system receives the deletion request and controls the target thread to perform the operation of deleting the target directory from the recycle bin based on the deletion request. The backup metadata system stores the deletion request sent by the primary metadata system and switches the backup metadata system to the primary metadata system when the primary metadata system malfunctions, so that the backup metadata system can be used as the switched primary metadata system to perform the operations that the primary metadata system has not completed.

10. A file erasing apparatus characterized by comprising: include: A receiving unit is configured to receive at least one deletion request from a client via a master metadata system, wherein the deletion request is used to instruct the deletion of a target directory within the recycle bin; The judgment unit is used to mark the target directory in the recycle bin as deleted through the master metadata system, and to determine whether the status of the recycle bin is a no-task state, wherein the no-task state means that there is no deletion operation in the recycle bin; An execution unit is configured to, when the recycle bin is in a no-task state, control the target thread of the master metadata system to execute the deletion operation corresponding to each deletion request, update the recycle bin's state to a task processing state, and record the cursor and operation identifier of the deletion operation. The cursor is used to record the number of files and bytes currently deleted, and the position of the file being deleted. The operation identifier is an index for the client to query the deletion progress of the deletion operation. The update unit is used to update the status of the recycle bin to a no-task state when the main metadata system detects that all files in the target directory have been deleted as indicated by the cursors of all threads.

11. A processor, comprising: The processor is used to run a program, wherein the program executes the file deletion method according to any one of claims 1 to 8.

12. An electronic device, comprising: The device includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the file deletion method according to any one of claims 1 to 8.

Images