File processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a file processing method and device, electronic equipment and a readable storage medium. According to the embodiment of the invention, meta-information, including a time index of a predetermined file and a file type of the predetermined file, of the predetermined file is obtained, a to-be-cleaned file is determined according to a life cycle in the time index of the predetermined file, and then the to-be-cleaned file is cleaned according to the file type of the to-be-cleaned file. Therefore, in the embodiment of the invention, the time index is set for the file with the life cycle, so that whether the file is the to-be-cleaned file or not can be quickly determined only by acquiring the time index of the file during file cleaning, the to-be-cleaned file is convenient to clean, the file cleaning efficiency is effectively improved, and the consumption of computing resources is reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly, to a file processing method, apparatus, electronic device, and readable storage medium. Background Art

[0002] In data services, various storage protocols correspond to different file structure storage methods, which makes it difficult to clean up files to be cleaned according to a unified rule. At the same time, the number of files is huge. Therefore, cleaning up files to be cleaned consumes a large amount of computing resources, and the cleaning efficiency of files to be cleaned is low. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a file processing method, apparatus, electronic device, and readable storage medium, which determine whether a file is a file to be cleaned according to the time index of the acquired file when cleaning up files, so as to facilitate cleaning up files to be cleaned, effectively improve the file cleaning efficiency, and reduce the consumption of computing resources.

[0004] In a first aspect, an embodiment of the present invention provides a file processing method, the method including:

[0005] Obtain meta-information of a predetermined file, where the meta-information includes a time index of the predetermined file and a file type of the predetermined file, and the time index includes a life cycle of the predetermined file;

[0006] Determine a file to be cleaned according to the time index;

[0007] Perform a cleaning process on the file to be cleaned according to the file type of the file to be cleaned.

[0008] Optionally, the determining a file to be cleaned according to the time index includes:

[0009] Determine whether the corresponding predetermined file has expired according to the life cycle;

[0010] Determine the expired predetermined file as the file to be cleaned.

[0011] Optionally, the performing a cleaning process on the file to be cleaned according to the file type of the file to be cleaned includes:

[0012] In response to the file type of the file to be cleaned being a junk file, move the file to be cleaned to the recycle bin and delete the file to be cleaned;

[0013] In response to the file type of the file to be cleaned being a directory file, delete the meta-information of the file to be cleaned;

[0014] In response to the file type of the file to be cleaned being a data file, move the file to be cleaned to the recycle bin.

[0015] Optionally, the cleaning process of the file to be cleaned according to the file type of the file to be cleaned further includes:

[0016] In response to the file type of the file to be cleaned being a directory file and the meta information of the file to be cleaned being unable to be deleted, move each file under the file to be cleaned to the recycle bin, and delete the meta information of the file to be cleaned.

[0017] Optionally, the cleaning process of the file to be cleaned according to the file type of the file to be cleaned further includes:

[0018] Delete the file to be cleaned in the recycle bin and the meta information of the file to be cleaned.

[0019] Optionally, the obtaining of the meta information of the predetermined file includes:

[0020] Obtain the meta information of the predetermined file according to the first period.

[0021] Optionally, the cleaning process of the file to be cleaned according to the file type of the file to be cleaned includes:

[0022] Clean a predetermined number of the files to be cleaned according to the file type of the file to be cleaned according to the second period.

[0023] Optionally, the method further includes:

[0024] Delete the mapping relationship corresponding to the data volume.

[0025] Optionally, the method further includes:

[0026] Receive a file creation request;

[0027] Determine whether to create a time index for the newly created file according to the request header of the file creation request.

[0028] Optionally, the determining whether to create a time index for the newly created file according to the request header of the file creation request includes:

[0029] In response to the request header including an expiration field, create a time index for the newly created file according to the expiration field.

[0030] Optionally, the method further includes:

[0031] In response to receiving a data access request for the file to be cleaned, return a query result according to the file type of the file to be cleaned.

[0032] Optionally, the returning of the query result according to the file type of the file to be cleaned includes:

[0033] In response to the file type of the file to be cleaned being a data file, returning an error prompt;

[0034] In response to the file type of the file to be cleaned being a directory file, returning the data of the file to be cleaned.

[0035] Optionally, the method further includes:

[0036] In response to receiving a rename instruction for a target file, modifying the life cycle of the target file according to the file type of the target file and the target parent node of the target file.

[0037] Optionally, the modifying of the life cycle of the target file according to the file type of the target file and the target parent node of the target file includes:

[0038] Determining the effective time corresponding to the target parent node;

[0039] In response to the file type of the target file being a data file, modifying the life cycle of the target file to the effective time;

[0040] In response to the file type of the target file being a directory file, modifying the life cycle of the target file to the effective time and keeping the life cycles of the files under the target file unchanged.

[0041] In a second aspect, an embodiment of the present invention provides a data storage system, the data storage system includes:

[0042] A meta - information service module, configured to store and manage the meta - information of files, the meta - information includes the time index of the file and the file type of the predetermined file, and the time index includes the life cycle of the file;

[0043] A storage interface service module with multiple storage interfaces, configured to call the meta - information service module to request the meta - information of the corresponding predetermined file, determine the file to be cleaned according to the time index of the predetermined file, and perform a cleaning process on the file to be cleaned according to the file type of the file to be cleaned.

[0044] In a third aspect, an embodiment of the present invention provides a file processing device, the device includes:

[0045] A meta - information acquisition unit, configured to acquire the meta - information of a predetermined file, the meta - information includes the time index of the predetermined file and the file type of the predetermined file, and the time index includes the life cycle of the predetermined file;

[0046] A file determination unit, configured to determine a file to be cleared according to the time index.

[0047] A file cleaning unit, configured to clean the file to be cleared according to the file type of the file to be cleared.

[0048] In a fourth aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, where the memory is used to store one or more computer program instructions, and wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of the first aspects.

[0049] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and the computer program, when executed by a processor, implements the method according to any one of the first aspects.

[0050] An embodiment of the present invention obtains metadata of a predetermined file including a time index of the predetermined file and a file type of the predetermined file, determines a file to be cleared according to a life cycle in the time index of the predetermined file, and further cleans the file to be cleared according to the file type of the file to be cleared. Thus, in an embodiment of the present invention, a file with a life cycle is set with a time index. Therefore, when cleaning files, it is only necessary to obtain the time index of the file to quickly determine whether the file is a file to be cleared, so as to facilitate cleaning the file to be cleared, effectively improving the file cleaning efficiency and reducing the consumption of computing resources. Description of the Drawings

[0051] Through the following description of the embodiments of the present invention with reference to the drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0052] Figure 1 is a schematic diagram of a data storage system according to an embodiment of the present invention;

[0053] Figure 2 is a flowchart of a file processing method according to an embodiment of the present invention;

[0054] Figure 3 is a schematic diagram of a file processing method according to an embodiment of the present invention;

[0055] Figure 4 is a flowchart of another file processing method according to an embodiment of the present invention;

[0056] Figure 5 is a schematic diagram of another file processing method according to an embodiment of the present invention;

[0057] Figure 6 is a flowchart of another file processing method according to an embodiment of the present invention;

[0058] Figure 7 is a flowchart of another file processing method according to an embodiment of the present invention;

[0059] Figure 8 is a schematic diagram of renaming a data file in another file processing method according to an embodiment of the present invention;

[0060] Figure 9 is a schematic diagram of renaming a directory file in another file processing method according to an embodiment of the present invention;

[0061] Figure 10 is a schematic diagram of a file processing apparatus according to an embodiment of the present invention;

[0062] Figure 11 is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed implementation manners

[0063] The following describes the present application based on embodiments, but the present application is not limited to these embodiments. In the following detailed description of the present application, some specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, procedures, elements, and circuits are not described in detail.

[0064] In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only, and the drawings are not necessarily drawn to scale.

[0065] Unless the context clearly requires otherwise, words such as "including" and "comprising" in the entire application document should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, it is the meaning of "including but not limited to".

[0066] In the description of the present application, it should be understood that terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0067] This embodiment provides a data storage system, which provides a unified data storage entry for various computing applications in the data lake ecosystem and integrates various storage protocols to support storage semantic fusion of multiple storage services such as Hadoop storage services (e.g., storage services based on HDFS distributed file system and HBase distributed NoSQL database), S3 (S3 Simple Storage Service), K8S CSI storage service, and Posix (Portable Operating System Interface of UNIX). It is applied to multiple data ecosystems and application scenarios, improving data access performance, reducing the storage space required for data applications. Moreover, when cleaning files, only the time index of the file needs to be obtained to quickly determine whether the file is a file to be cleaned, facilitating the cleaning of files to be cleaned, effectively improving file cleaning efficiency, and reducing the consumption of computing resources.

[0068] Figure 1 is a schematic diagram of the data storage system according to an embodiment of the present invention. As Figure 1 shown, the data storage system 20 of this embodiment is connected to the service layer 10 and the storage layer 30 to determine whether a file is a file to be cleaned according to the life cycle in the time index of the file and clean the file to be cleaned.

[0069] In this embodiment, the data storage system 20 performs semantic fusion on various storage services so that each service layer 10 can call an appropriate storage interface to operate on data files or the data therein (such as read, write, delete, etc.). Among them, the data storage system 20 includes a storage interface service module 21 with multiple storage interfaces and a meta-information service module 22. Optionally, the storage interface service module 21 of the data storage system 20 may include interface services such as Posix file interface, HDFS SDK, CSI, S3 interface, S2 interface, and image processing interface. It should be understood that this embodiment is not limited to the above storage interfaces, and other storage interfaces for implementing storage services can also be integrated into this embodiment.

[0070] In this embodiment, regardless of which storage protocol (such as S3 or file system, etc.), the data file consists of two parts: meta-information and data. Among them, the meta-information is stored in the meta-information service, and the data is stored in the corresponding data storage space (such as GIFT DFS storage system, S3 storage system, OSS storage system, COS storage system, etc.).

[0071] Further, in this embodiment, the meta - information service module 22 is configured to store and manage the meta - information of files. Among them, the meta - information of files includes the time index of the file and the file type of the file, and the time index of the file includes the life cycle of the file.

[0072] Thus, in this embodiment, the storage interface service module 21 is further configured to call the meta - information service module 22 to request the meta - information of a corresponding predetermined file, determine the files to be cleaned according to the time index of the predetermined file, and perform cleaning processing on the files to be cleaned according to the file types of the files to be cleaned.

[0073] Further, the data storage system of this embodiment can be implemented based on a single server or a server cluster, and this embodiment does not limit this.

[0074] Figure 2 is a flowchart of a file processing method according to an embodiment of the present invention. As Figure 2 shown, the file processing method according to the embodiment of the present invention includes the following steps:

[0075] Step S110, obtain the meta - information of a predetermined file.

[0076] According to the user's usage requirements for files, files can be divided into short - term storage files with a life cycle and permanent storage files. During the file creation process, the meta - information service of the data storage system can determine whether a file is a file with a life cycle according to the user's settings. If the file is a file with a life cycle, the meta - information service of the data storage system will create a time index including the life cycle of the file for the file and store the time index in the meta - information service as a part of the meta - information of the file. Optionally, the time index also includes the absolute path of the file.

[0077] In this embodiment, different data files or directory files can be stored through a data volume. Therefore, a data volume can correspond to multiple time indexes. The time index of the data volume includes the effective sub - path of the data volume and the life cycle of the data volume under this effective sub - path. The life cycle can be set to 1 day, 7 days, 15 days, 30 days, etc. The effective sub - path is the absolute path of the sub - directory corresponding to the data volume, such as / a / , / b / , / a / b / c / , etc. The sub - directories in different time indexes cannot cross - overlap.

[0078] For example, the time index corresponding to the data volume v1 includes Index 1: effective sub-path - / a / b / c / , lifecycle - 30 days, that is, the lifecycle of the files corresponding to the effective sub-path / a / b / c / is 30 days; Index 2: effective sub-path - / e / , lifecycle - 7 days, that is, the lifecycle of the files corresponding to the effective sub-path - / e / is 7 days; Index 3: effective sub-path - / d / a / , lifecycle 1 day, that is, the lifecycle of the files corresponding to the effective sub-path / d / a / is 1 day; Index 4: effective sub-path - / c / , lifecycle - 90 days, that is, the lifecycle of the files corresponding to the effective sub-path / c / is 90 days.

[0079] In step S110, the data storage system obtains the meta-information of each predetermined file through the remove service in the storage interface service module to obtain the time index in the meta-information. The remove service obtains the time index of each predetermined file by calling the meta-information service.

[0080] Optionally, the remove service in the storage interface service module scans the time indexes of each predetermined file in the order of the expiration time of each predetermined file. The expiration time is determined according to the creation time and lifecycle of the predetermined file. For example, if the creation time of file F1 is November 20, 2023 and the lifecycle is 7 days, then the expiration time of file F1 is November 27, 2023.

[0081] Furthermore, the remove service in the storage interface service module can obtain the meta-information of the predetermined file according to a preset first period. For example, it can be set to scan the time indexes of each predetermined file once a day, scan the time indexes of each predetermined file once every other day, etc.

[0082] To avoid affecting the response time of the data storage system during the peak period of data storage system access, the first period can be adjusted according to actual needs. For example, during the peak access period, set to obtain the meta-information of the predetermined file once every three days, and during the non-peak access period, set to obtain it once a day.

[0083] Step S120, determining the files to be cleaned according to the time index.

[0084] After obtaining the time index of the predetermined file, the data storage system can determine the files to be cleaned according to the lifecycle of each predetermined file. For each predetermined file, the remove service in the storage interface service module can judge whether the predetermined file has expired according to the current scan time and the expiration time of the predetermined file. If the file has expired, the remove service will determine the expired predetermined file as the file to be cleaned.

[0085] Step S130, perform a cleaning process on the file to be cleaned according to the file type of the file to be cleaned.

[0086] After determining the file to be cleaned, the data storage system will perform a cleaning process on the file to be cleaned in different ways according to the file type of the file to be cleaned.

[0087] Among them, if the file type of the file to be cleaned is a junk file, the trash service in the storage interface service module will move the junk file to the recycle bin, and the clean service will directly delete the junk file; if the file type of the file to be cleaned is a data file, in order to avoid the situation of accidental deletion of the data file or the possibility of subsequent data recovery, the removal service of the database storage system will move the file to be cleaned to the recycle bin instead of directly deleting it; if the file type of the file to be cleaned is a directory file, the removal service of the database storage system will call the meta-information service to delete the meta-information of the file to be cleaned.

[0088] If the file type of the file to be cleaned is a directory file, and the removal service cannot automatically directly delete the meta-information of the file to be cleaned when the corresponding directory file expires, the removal service will move each file under the file to be cleaned to the recycle bin, and then actively delete the meta-information of the file to be cleaned.

[0089] In an optional implementation manner, the file storage system can establish an association relationship between the file to be cleaned and the recycle bin, and delete the association relationship between the file to be cleaned and its folder in the meta-information service, so as to achieve the purpose of moving the file to be cleaned to the recycle bin.

[0090] Specifically, the storage interface service module will call the meta-information service module to request and obtain the index identifier (node_id) of the parent node of the file to be cleaned, and determine the folder where the file to be cleaned is located according to the index identifier of the parent node of the file to be cleaned, and then delete the association relationship between the file to be cleaned and its folder in the meta-information service. After moving the file to be cleaned to the corresponding recycle bin, the storage interface service module will establish an association relationship between the file to be cleaned and the recycle bin in the meta-information service to update the meta-information of the file to be cleaned.

[0091] For example, if the file to be cleaned F2 is located in folder A, then the association relationship between the file to be cleaned F2 and folder A is stored in the meta-information service. During the cleaning process of the file to be cleaned F2, the storage interface service module deletes the association relationship between the file to be cleaned F2 and folder A, and establishes an association relationship between the file to be cleaned F2 and the recycle bin, that is, moves the file to be cleaned F2 from folder A to the recycle bin.

[0092] Further, the cleaning service can clean the files to be cleaned according to the file types of the files to be cleaned in a preset second period, and the cleaning service can clean a preset number of files to be cleaned in each second period. For example, the cleaning service can clean 100 files to be cleaned every day according to the file types of the files to be cleaned.

[0093] To avoid affecting the response time of the data storage system during the peak period of data storage system access, the second period and the preset number can be adjusted according to actual needs. For example, during the peak access period, set to clean 100 files to be cleaned every three days, and during the non-peak access period, set to clean 300 files to be cleaned every day.

[0094] It is easy to understand that in the embodiments of the present invention, the first period and the second period can be the same period or different periods, and the embodiments of the present invention do not limit this.

[0095] In this embodiment, the cleaning service of the file storage system scans the files to be cleaned in the recycle bin, deletes the files to be cleaned in the recycle bin, and then deletes the meta-information of each file to be cleaned in the recycle bin from the meta-information service to avoid the situation where the files to be cleaned can still be accessed by the client. Optionally, the cleaning service can also clean the meta-information of the files to be cleaned in the recycle bin in a preset second period.

[0096] Figure 3 is a schematic diagram of a file processing method according to an embodiment of the present invention. As Figure 3 shown, after the life cycle of the user configuration file, the meta-information service of the file storage system creates a time index for the file according to the life cycle set by the user, and stores the time index of the file in the meta-information service module 31. The Ttl_remove service (that is, the cleaning service) 33 scans the time index of the files in the meta-information service module 31, and determines the files to be cleaned according to the time index of each file. After determining the files to be cleaned, for data files, the ttl_remove service 33 moves the data files to the recycle bin 32; for directory files, the ttl_remove service 33 directly deletes the meta-information of the directory files. The Trash service 34 scans the meta-information service module 31 and moves the junk files to the recycle bin 32. The Ttl_clean service 35 scans the recycle bin 32. For junk files, the ttl_clean service 35 directly deletes the junk files after they are moved to the recycle bin 32; for data files, the ttl_clean service 35 deletes the meta-information of the data files.

[0097] In an optional implementation manner, the method in this embodiment may further include: deleting the mapping relationship corresponding to the data volume.

[0098] The file storage system provides a unified data storage entry for various computing applications in the data lake ecosystem. However, the index information of the directory volume used to store data files or directory files in the storage interface service may be different from its index information in the data storage entry. Therefore, in order to enable users to access the files in the file storage information through the unified data storage entry, for the data volume, the file storage system stores the mapping relationship between the index information of the data volume in the unified data storage entry and the index information of the data volume in different storage interface services in a predetermined database (for example, levelDB). After the data volume is cleared, that is, after the storage space corresponding to the data volume is released, the file storage system deletes the mapping relationship corresponding to the data volume from the predetermined database.

[0099] The embodiment of the present invention obtains the meta-information of a predetermined file including the time index of the predetermined file and the file type of the predetermined file, determines the files to be cleared according to the life cycle in the time index of the predetermined file, and then performs a clearing process on the files to be cleared according to the file type of the files to be cleared. In the embodiment of the present invention, files with a life cycle are set with a time index, while permanently stored files are not set with a time index. Thus, when performing file clearing, it is not necessary to obtain the meta-information of all files. Only by obtaining the meta-information of the files with a life cycle can it be quickly determined whether the file is a file to be cleared according to the time index in the meta-information, so as to automatically clear the files to be cleared, effectively improving the file clearing efficiency and reducing the consumption of computing resources.

[0100] Figure 4 is a flowchart of another file processing method according to an embodiment of the present invention. As Figure 4 shown, the file processing method according to the embodiment of the present invention includes the following steps:

[0101] Step S210, receive a file creation request.

[0102] In this embodiment, the file storage system can receive a file creation request sent by a client. The file creation request is a type of RPC (Remote Procedure Call) request. For a newly created file with a life cycle set by the user, the client can add an expire field to the request header of the file creation request so that the meta-information service of the file storage system can create a time index for the newly created file. The expire field includes the life cycle of the newly created file and may also include the absolute path of the newly created file.

[0103] According to the different storage methods of the file structure, the client will use different methods to determine whether a newly created file is a file with a life cycle. In an alternative implementation, the storage method of the file structure corresponding to the newly created file is HDFS or S3. The prefix of the absolute path corresponding to the newly created file that uses this storage method and has a life cycle will include the LifeCycle.Prefix field. Therefore, when the prefix of the absolute path of a newly created file includes the LifeCycle.Prefix field, the client can determine that the newly created file is a file with a life cycle and add an expiration field to the request header of the file creation request corresponding to this newly created file.

[0104] In another alternative implementation, the storage method of the file structure corresponding to the newly created file is Posix. The client needs to determine whether the file using this storage method is a file with a life cycle. Specifically, the client needs to determine the device (such as a disk drive, printer, etc.) identifier and the index identifier of the parent node corresponding to the newly created file according to the index information (inode) of the newly created file, and obtain the directory tree corresponding to the newly created file from the Inode Tree cache data according to the index identifier of the parent node and the device identifier. Then, according to the index identifier of the newly created file, find the absolute path of the newly created file in the directory tree. Furthermore, when the prefix of the absolute path of the newly created file includes the LifeCycle.Prefix field, it is determined that the newly created file is a file with a life cycle.

[0105] Step S220: Determine whether to create a time index for the newly created file according to the request header of the file creation request.

[0106] After extracting the expiration field carried in the request header of the file creation request, the metadata service of the file storage system will create a time index for the newly created file corresponding to this file creation request according to this expiration field.

[0107] It is easy to understand that if the expiration field is not carried in the file creation request, the metadata service of the file storage system will not create a time index for the newly created file corresponding to this file creation request according to this expiration field.

[0108] In this embodiment, if the expiration field is not carried in the request header, the metadata service of the file storage system will not create a time index for the newly created file. In this way, only files with a life cycle will be created with a time index. Therefore, during the file cleaning process, as long as the time index of the file can be obtained, it can be determined that the file is a file with a life cycle, and it can be determined whether the file is a file to be cleaned according to the time index of the file. Compared with the existing method in which all file metadata needs to be obtained to determine all files to be cleaned, this embodiment can effectively reduce the consumption of computing resources.

[0109] Figure 5 It is a schematic diagram of another file processing method according to an embodiment of the present invention. As Figure 5 shown, when the client creates file B, it creates the device file mknod() corresponding to file B according to the life cycle 51 of file B, the directory tree 52, the index identifier of the parent node corresponding to file B, and the device identifier, and then sends a file creation request carrying the life cycle of file B to the file storage system. The file storage system stores the life cycle of file B in the meta-information service 53, so that the meta-information service 53 can call the time index 54 for creating file B at the time of creating file B.

[0110] Step S230, obtain the meta-information of the predetermined file.

[0111] In this embodiment, the implementation manners of step S230 and step S110 are similar, and will not be described in detail here.

[0112] Step S240, determine the files to be cleaned according to the time index.

[0113] In this embodiment, the implementation manners of step S240 and step S120 are similar, and will not be described in detail here.

[0114] Step S250, perform cleaning processing on the files to be cleaned according to the file types of the files to be cleaned.

[0115] In this embodiment, the implementation manners of step S250 and step S130 are similar, and will not be described in detail here.

[0116] The embodiment of the present invention receives a file creation request, determines whether to create a time index for a newly created file according to the request header of the file creation request, obtains the meta-information of the predetermined file including the time index of the predetermined file and the file type of the predetermined file, determines the files to be cleaned according to the life cycle in the time index of the predetermined file, and further performs cleaning processing on the files to be cleaned according to the file types of the files to be cleaned. In the embodiment of the present invention, a time index for the file will be created only when the request header of the file creation request includes an expiration field, which enables a file with a life cycle to be set with a time index, while a permanently stored file will not be set with a time index. Thus, when performing file cleaning, it is not necessary to obtain the meta-information of all files, and only the meta-information of the files with a life cycle needs to be obtained, and it is possible to quickly determine whether the file is a file to be cleaned according to the time index in the meta-information, so as to facilitate the automatic cleaning of the files to be cleaned, effectively improving the file cleaning efficiency and reducing the consumption of computing resources.

[0117] Figure 6 It is a flowchart of another file processing method according to an embodiment of the present invention. As Figure 6 shown, the file processing method according to the embodiment of the present invention includes the following steps:

[0118] Step S310: Obtain the meta-information of a predetermined file.

[0119] In this embodiment, the implementation manners of step S310 and step S110 are similar and will not be elaborated here.

[0120] Step S320: Determine the files to be cleaned according to the time index.

[0121] In this embodiment, the implementation manners of step S320 and step S120 are similar and will not be elaborated here.

[0122] Step S330: Clean the files to be cleaned according to the file types of the files to be cleaned.

[0123] In this embodiment, the implementation manners of step S330 and step S130 are similar and will not be elaborated here.

[0124] Step S340: In response to receiving a data access request for the files to be cleaned, return a query result according to the file types of the files to be cleaned.

[0125] After receiving a data access request for the files to be cleaned sent by the client, the file storage system will return different query results according to the types of the files to be cleaned. When the file type of the file to be cleaned is a data file, since the data file has expired and cannot be accessed, the file storage system will return an error prompt, such as Error NoEntry; when the file type of the file to be cleaned is a log file, the file storage system will normally return the data.

[0126] The embodiment of the present invention obtains the meta-information of a predetermined file including the time index of the predetermined file and the file type of the predetermined file, determines the files to be cleaned according to the life cycle in the time index of the predetermined file, and then cleans the files to be cleaned according to the file types of the files to be cleaned. At the same time, after receiving a data access request for the files to be cleaned, a query result is returned according to the file types of the files to be cleaned. In the embodiment of the present invention, files with a life cycle are set with a time index, while permanently stored files are not set with a time index. Thus, when cleaning files, it is not necessary to obtain the meta-information of all files. Only by obtaining the meta-information of files with a life cycle, it is possible to quickly determine whether the file is a file to be cleaned according to the time index in the meta-information, so as to automatically clean the files to be cleaned, effectively improving the file cleaning efficiency and reducing the consumption of computing resources. At the same time, an error prompt is returned for expired data files to prompt the user that the data files cannot be accessed.

[0127] Figure 7 is a flowchart of another file processing method according to an embodiment of the present invention. As Figure 7As shown in the figure, the file processing method according to an embodiment of the present invention includes the following steps:

[0128] Step S410: Obtain the meta information of a predetermined file.

[0129] In this embodiment, the implementation manners of step S410 and step S110 are similar, and will not be described herein again.

[0130] Step S420: Determine the files to be cleaned according to the time index.

[0131] In this embodiment, the implementation manners of step S420 and step S120 are similar, and will not be described herein again.

[0132] Step S430: Clean the files to be cleaned according to the file types of the files to be cleaned.

[0133] In this embodiment, the implementation manners of step S430 and step S130 are similar, and will not be described herein again.

[0134] Step S440: In response to receiving a rename instruction for a target file, modify the life cycle of the target file according to the file type of the target file and the target parent node of the target file.

[0135] According to actual usage requirements, the user can modify the absolute path of the file by renaming. After the file storage system receives the rename instruction for the target file sent by the client, it modifies the parent node of the target file in the meta information service to the target parent node, and modifies the life cycle of the target file according to the target parent node of the target file and the file type of the target file.

[0136] In this step, the file storage system determines the target parent node of the target file according to the rename instruction, and then determines the effective time of this parent node. When the file type of the target file is a data file, the file storage system modifies the life cycle of the target file to this effective time in the meta information service; when the file type of the target file is a directory file, the file storage system modifies the life cycle of the target file to this effective time in the meta information service, and at the same time keeps the life cycles of the files under the target file unchanged.

[0137] Figure 8 It is a schematic diagram of renaming a data file in another file processing method according to an embodiment of the present invention. As Figure 8As shown in the figure, file m and file n are target files of the data file type. Node c is the current parent node of file m and file n, and the effective time of node c is 7 days. After the file storage system receives the rename instruction for file m and file n, it determines that node d is the target parent node of file m and file n according to the rename instruction, and determines that the effective time of node d is 1 day. Then, it modifies the parent node of file m and file n to node d, and modifies the life cycle of both file m and file n to the effective time of node d, that is, 1 day.

[0138] Figure 9 It is a schematic diagram of renaming a directory file in another file processing method according to an embodiment of the present invention. As Figure 9 shown, the file 91 within the dashed box is the directory file of file x and file y. According to node e, it can be determined that the effective times of file 91, file x, and file y are all 7 days. After the file storage system receives the rename instruction for file 91, it determines that node f is the target parent node of file 91 according to the rename instruction, and determines that the effective time of node f is 1 day. Then, it modifies the parent node of node e in file 91 to node f, and modifies the life cycle of file 91 from the effective time of node e to the effective time of node f, that is, from 7 days to 1 day, while keeping the life cycles of file x and file y unchanged, still 1 day.

[0139] The embodiment of the present invention obtains the meta-information of a predetermined file including the time index of the predetermined file and the file type of the predetermined file, determines the file to be cleaned according to the life cycle in the time index of the predetermined file, and then performs cleaning processing on the file to be cleaned according to the file type of the file to be cleaned. At the same time, after receiving the rename instruction for the target file, it modifies the life cycle of the target file according to the file type of the file to be cleaned and the target parent node of the target file. In the embodiment of the present invention, files with a life cycle are set with a time index, while permanently stored files are not set with a time index. Thus, when performing file cleaning, it is not necessary to obtain the meta-information of all files. Only by obtaining the meta-information of files with a life cycle, it is possible to quickly determine whether the file is a file to be cleaned according to the time index in the meta-information, so as to automatically clean the file to be cleaned, effectively improving the file cleaning efficiency and reducing the consumption of computing resources. At the same time, it modifies the life cycle of the file according to the rename instruction of the file to synchronize the modification of the life cycle when the absolute path of the file changes.

[0140] Figure 10 It is a schematic diagram of the file processing device according to an embodiment of the present invention. As Figure 10 shown, the file processing device according to the embodiment of the present invention includes a meta-information acquisition unit 1001, a file determination unit 1002, and a file cleaning unit 1003.

[0141] Among them, the meta - information acquisition unit 1001 is used to acquire the meta - information of a predetermined file, and the meta - information includes the time index of the predetermined file and the file type of the predetermined file, and the time index includes the life cycle of the predetermined file. The file determination unit 1002 is used to determine the files to be cleaned according to the time index. The file cleaning unit 1003 is used to perform cleaning processing on the files to be cleaned according to the file type of the files to be cleaned.

[0142] In an optional implementation manner, the file determination unit 1002 includes a first determination subunit and a second determination subunit.

[0143] Among them, the first determination subunit is used to determine whether the corresponding predetermined file has expired according to the life cycle. The second determination subunit is used to determine the expired predetermined files as the files to be cleaned.

[0144] In an optional implementation manner, the file cleaning unit 1003 includes a first cleaning subunit, a second cleaning subunit, and a third cleaning subunit.

[0145] Among them, the first cleaning subunit is used to move the file to be cleaned to the recycle bin and delete the file to be cleaned in response to the file type of the file to be cleaned being a junk file. The second cleaning subunit is used to delete the meta - information of the file to be cleaned in response to the file type of the file to be cleaned being a directory file. The third cleaning subunit is used to move the file to be cleaned to the recycle bin in response to the file type of the file to be cleaned being a data file.

[0146] In an optional implementation manner, the file cleaning unit 1003 further includes a fourth cleaning subunit.

[0147] Among them, the fourth cleaning subunit is used to move each file under the file to be cleaned to the recycle bin and delete the meta - information of the file to be cleaned in response to the file type of the file to be cleaned being a directory file and the meta - information of the file to be cleaned cannot be deleted.

[0148] In an optional implementation manner, the file cleaning unit 1003 further includes a fifth cleaning subunit.

[0149] Among them, the fifth cleaning subunit is used to delete the file to be cleaned in the recycle bin and the meta - information of the file to be cleaned.

[0150] In an optional implementation manner, the meta - information acquisition unit 1001 is used to acquire the meta - information of the predetermined file according to a first period.

[0151] In an optional implementation manner, the file cleaning unit 1003 includes a sixth cleaning subunit.

[0152] Among them, the sixth cleaning subunit is used to clean a predetermined number of the files to be cleaned according to the file type of the files to be cleaned in the second cycle.

[0153] In an optional implementation, the file processing device further includes a mapping relationship deletion unit.

[0154] Among them, the mapping relationship deletion unit is used to delete the mapping relationship corresponding to the data volume.

[0155] In an optional implementation, the file processing device further includes a request receiving unit and an index creation unit.

[0156] Among them, the request receiving unit is used to receive a file creation request. The index creation unit is used to determine whether to create a time index for the newly created file according to the request header of the file creation request.

[0157] In an optional implementation, the index creation unit is used to create the time index for the newly created file according to the expiration field in response to the request header including the expiration field.

[0158] In an optional implementation, the file processing device further includes a result return unit.

[0159] Among them, the result return unit is used to return a query result according to the file type of the file to be cleaned in response to receiving a data access request for the file to be cleaned.

[0160] In an optional implementation, the result return unit includes a first return subunit and a second return subunit.

[0161] Among them, the first return subunit is used to return an error prompt in response to the file type of the file to be cleaned being a data file. The second return subunit is used to return the data of the file to be cleaned in response to the file type of the file to be cleaned being a directory file.

[0162] In an optional implementation, the file processing device further includes a renaming unit.

[0163] Among them, the renaming unit is used to modify the life cycle of the target file according to the file type of the target file and the target parent node of the target file in response to receiving a renaming instruction for the target file.

[0164] In an optional implementation, the renaming unit includes a time determination subunit, a first modification subunit, and a second modification subunit.

[0165] Among them, the time determination subunit is used to determine the effective time corresponding to the target parent node. The first modification subunit is used to modify the lifecycle of the target file to the effective time in response to the file type of the target file being a data file. The second modification subunit is used to modify the lifecycle of the target file to the effective time in response to the file type of the target file being a directory file, and keep the lifecycles of the files under the target file unchanged.

[0166] The embodiment of the present invention obtains the meta-information of a predetermined file including the time index of the predetermined file and the file type of the predetermined file, determines the file to be cleaned according to the lifecycle in the time index of the predetermined file, and then performs cleaning processing on the file to be cleaned according to the file type of the file to be cleaned. In the embodiment of the present invention, a file with a lifecycle is set with a time index, while a permanently stored file is not set with a time index. Thus, when cleaning files, it is not necessary to obtain the meta-information of all files. Only by obtaining the meta-information of the files with a lifecycle, it is possible to quickly determine whether the file is a file to be cleaned according to the time index in the meta-information, so as to automatically clean the file to be cleaned, effectively improving the file cleaning efficiency and reducing the consumption of computing resources.

[0167] Figure 11 is a schematic diagram of the electronic device according to the embodiment of the present invention. As Figure 11 shown, the electronic device 11 is a general data processing device, which includes a general computer hardware structure, and at least includes a processor 1101 and a memory 1102. The processor 1101 and the memory 1102 are connected through a bus 1103. The memory 1102 is suitable for storing instructions or programs executable by the processor 1101. The processor 1101 can be an independent microprocessor or a set of one or more microprocessors. Thus, the processor 1101 executes the instructions stored in the memory 1102, thereby executing the method flow of the embodiment of the present invention as described above to implement the processing of data and the control of other devices. The bus 1103 connects the above-mentioned multiple components together, and at the same time connects the above-mentioned components to a display controller 1104, a display device, and an input / output (I / O) device 1105. The input / output (I / O) device 1105 can be a mouse, a keyboard, a modem, a network interface, a touch input device, a somatosensory input device, a printer, and other devices well known in the art. Typically, the input / output (I / O) device 1105 is connected to the system through an input / output (I / O) controller 1106.

[0168] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, an apparatus (device), or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be implemented as a computer program product on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0169] The present application is described with reference to the flowcharts of methods, apparatuses (devices), and computer program products according to the embodiments of the present application. It should be understood that each process in the flowchart can be implemented by computer program instructions.

[0170] These computer program instructions can be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device that implements the process Figure 1 specified functions in one or more of these processes.

[0171] These computer program instructions can also be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices produce a device for implementing the Figure 1 specified functions in one or more of these processes.

[0172] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, and the computer-readable program is used for a computer to execute the above-mentioned partial or all method embodiments.

[0173] That is, those skilled in the art can understand that all or part of the steps in implementing the above-mentioned method embodiments can be completed by specifying relevant hardware through a program. The program is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0174] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A file processing method, characterized in that, the method includes: obtaining metadata of a predetermined file, where the metadata includes a time index of the predetermined file and a file type of the predetermined file, and the time index includes a life cycle of the predetermined file; determining a file to be cleaned up according to the time index; performing a cleanup process on the file to be cleaned up according to the file type of the file to be cleaned up.

2. The method according to claim 1, characterized in that, the determining a file to be cleaned up according to the time index includes: determining whether the corresponding predetermined file has expired according to the life cycle; determining the expired predetermined file as the file to be cleaned up.

3. The method according to claim 1, characterized in that, the performing a cleanup process on the file to be cleaned up according to the file type of the file to be cleaned up includes: in response to the file type of the file to be cleaned up being a junk file, moving the file to be cleaned up to the recycle bin and deleting the file to be cleaned up; in response to the file type of the file to be cleaned up being a directory file, deleting the metadata of the file to be cleaned up; in response to the file type of the file to be cleaned up being a data file, moving the file to be cleaned up to the recycle bin.

4. The method according to claim 3, characterized in that, the performing a cleanup process on the file to be cleaned up according to the file type of the file to be cleaned up further includes: in response to the file type of the file to be cleaned up being a directory file and the metadata of the file to be cleaned up cannot be deleted, moving each file under the file to be cleaned up to the recycle bin and deleting the metadata of the file to be cleaned up.

5. The method according to claim 3 or 4, characterized in that, the performing a cleanup process on the file to be cleaned up according to the file type of the file to be cleaned up further includes: deleting the file to be cleaned up in the recycle bin and the metadata of the file to be cleaned up.

6. The method according to claim 1, characterized in that, the obtaining metadata of a predetermined file includes: obtaining the metadata of the predetermined file according to a first period.

7. The method according to claim 6, characterized in that, the performing a cleanup process on the file to be cleaned up according to the file type of the file to be cleaned up includes: performing a cleanup process on a predetermined number of files to be cleaned up according to the file type of the file to be cleaned up according to a second period.

8. The method according to claim 1, characterized in that, the method further includes: deleting the mapping relationship corresponding to the data volume.

9. The method according to claim 1 or 8, characterized in that, the method further includes: receiving a file creation request; determining whether to create a time index for a new file according to the request header of the file creation request.

10. The method according to claim 9, characterized in that, the determining whether to create a time index for a new file according to the request header of the file creation request includes: in response to the request header including an expiration field, creating a time index for the new file according to the expiration field.

11. The method according to any one of claims 1, 8 or 9, characterized in that, The method further includes: In response to receiving a data access request for a file to be cleaned up, returning a query result according to the file type of the file to be cleaned up.

12. The method according to claim 11, wherein, The returning a query result according to the file type of the file to be cleaned up includes: In response to the file type of the file to be cleaned up being a data file, returning an error prompt; In response to the file type of the file to be cleaned up being a directory file, returning the data of the file to be cleaned up.

13. The method according to any one of claims 1, 8, 9, or 11, wherein, The method further includes: In response to receiving a rename instruction for a target file, modifying the life cycle of the target file according to the file type of the target file and the target parent node of the target file.

14. The method according to claim 13, wherein, The modifying the life cycle of the target file according to the file type of the target file and the target parent node of the target file includes: Determining the effective time corresponding to the target parent node; In response to the file type of the target file being a data file, modifying the life cycle of the target file to the effective time; In response to the file type of the target file being a directory file, modifying the life cycle of the target file to the effective time and keeping the life cycles of the files under the target file unchanged.

15. A data storage system, wherein, The data storage system includes: A meta-information service module, configured to store and manage the meta-information of files, the meta-information including the time index of the file and the file type of the file, and the time index including the life cycle of the file; A storage interface service module having a plurality of storage interfaces, configured to call the meta-information service module to request the meta-information of a corresponding predetermined file, determine a file to be cleaned up according to the time index of the predetermined file, and perform a cleaning process on the file to be cleaned up according to the file type of the file to be cleaned up.

16. A file processing device, wherein, The device includes: A meta-information acquisition unit, configured to acquire the meta-information of a predetermined file, the meta-information including the time index of the predetermined file and the file type of the predetermined file, and the time index including the life cycle of the predetermined file; A file determination unit, configured to determine a file to be cleaned up according to the time index; A file cleaning unit, configured to perform a cleaning process on the file to be cleaned up according to the file type of the file to be cleaned up.

17. An electronic device, including a memory and a processor, wherein, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1-14.

18. A computer-readable storage medium, wherein, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1-14.

Images