File processing method and device, electronic equipment and storage medium

Abstract

The invention provides a file processing method and device, electronic equipment and a storage medium. In the embodiment of the invention, the difference between the expected deletion moments of the plurality of second files in the third file from the file system is smaller than the first preset threshold value, so that the remaining second files in the third file can be completely deleted within a short time after one second file in the third file is deleted for the first time; the voidage of the third file can reach 100%, so that the third file can be directly deleted from the file system and can not be re-merged (the second file does not exist in the third file), or the number of times of re-merging the big file obtained by merging is reduced, and the efficiency of re-merging the big file is improved. The reading and writing of the disk are reduced, the load of the file system and the disk loss are reduced, and the hole recovery efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a file processing method, device, electronic device and storage medium. Background Art

[0002] In an object storage system, each file is identified by a unique identifier and can contain metadata as well as actual data. This approach bundles data and its associated metadata (such as creation date, size, and type), making data storage, retrieval, and management more efficient and flexible.

[0003] However, if the object storage system stores a large number of small files (for example, files smaller than 1MB), problems will arise in terms of storage efficiency, read and write performance, and metadata management. Summary of the invention

[0004] The present application shows a file processing method, device, electronic device and storage medium.

[0005] In a first aspect, the present application shows a file processing method, which includes: screening multiple first files in a file system, the storage times of the multiple first files when they are respectively stored in the file system are the same; obtaining the deletion times at which the multiple first files are each expected to be deleted from the file system; screening multiple second files from the multiple first files, the difference between the deletion times at which the multiple second files are each expected to be deleted from the file system is less than a first preset threshold; merging the multiple second files into a third file; storing the third file in the file system; storing in the file system a first correspondence between the identifiers of each second file, the identifier of the third file, and the offset addresses of each second file in the third file; and deleting the multiple second files in the file system.

[0006] In a second aspect, the present application shows a file processing method, the method comprising: detecting a hole rate of a third file in a file system; wherein the third file is obtained by merging multiple second files and storing them in the file system, the multiple second files are screened from multiple first files in the file system, the difference between the deletion times when the multiple second files are expected to be deleted from the file system is less than a first preset threshold, and the storage times when the multiple first files are respectively stored in the file system are the same; in the case where a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; a first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file in the target third file is deleted in the file system; wherein the first correspondence is stored in the file system after the target third file is stored in the file system in advance.

[0007] In a third aspect, the present application shows a file processing device, which includes: a first screening module, used to screen multiple first files in a file system, and the storage times of the multiple first files when they are respectively stored in the file system are the same; a first acquisition module, used to obtain the deletion times when the multiple first files are each expected to be deleted from the file system; a second screening module, used to screen multiple second files from the multiple first files, and the difference between the deletion times when the multiple second files are each expected to be deleted from the file system is less than a first preset threshold; a first merging module, used to merge the multiple second files into a third file; a first storage module, used to store the third file in the file system; a second storage module, used to store in the file system a first correspondence between the identifiers of each second file, the identifier of the third file, and the offset addresses of each second file in the third file; a first deletion module, used to delete the multiple second files in the file system.

[0008] In a fourth aspect, the present application shows a file processing device, which includes: a detection module, which is used to detect the hole rate of a third file in a file system; wherein the third file is obtained by merging multiple second files and stored in the file system, the multiple second files are screened from multiple first files in the file system, the difference between the deletion times when the multiple second files are expected to be deleted from the file system is less than a first preset threshold, and the storage times when the multiple first files are respectively stored in the file system are the same; a second deletion module, which is used to delete the target third file in the file system when a target third file with a hole rate of 100% is detected in the file system; a third deletion module, which is used to delete the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file in the target third file in the file system; wherein the first correspondence is stored in the file system after the target third file is stored in the file system in advance.

[0009] In a fifth aspect, the present application illustrates an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the method described in any of the above aspects.

[0010] In a sixth aspect, the present application illustrates a non-temporary computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute a method as described in any of the above aspects.

[0011] In a seventh aspect, the present application illustrates a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is enabled to execute the method described in any of the above aspects.

[0012] The technical solution provided by this application may have the following beneficial effects:

[0013] In the present application, a plurality of first files are screened in a file system, and the storage times of the plurality of first files when they are respectively stored in the file system are the same; the deletion times at which the plurality of first files are respectively expected to be deleted from the file system are obtained; a plurality of second files are screened in the plurality of first files, and the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold; the plurality of second files are merged into a third file; the third file is stored in the file system; a first correspondence between the identifiers of the respective second files, the identifiers of the third file, and the offset addresses of the respective second files in the third file is stored in the file system; and the plurality of second files are deleted in the file system.

[0014] In this way, the hole rate of the third file in the file system can be detected; when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; and the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file are deleted in the file system.

[0015] Since the difference between the deletion times of the multiple second files in the third file when they are expected to be deleted from the file system is less than the first preset threshold, the remaining second files in the third file will be all deleted within a short period of time after a second file in the third file is deleted for the first time, so that the hole rate of the third file will reach 100%, so that the third file can be directly deleted from the file system, and there is no need to re-merge the third file (the second file no longer exists in the third file), or the number of times the merged large file needs to be re-merged is reduced, the disk reading and writing is reduced, the file system load and disk loss are reduced, and the hole recovery efficiency is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of the steps of a file processing method of the present application.

[0017] Figure 2 It is a flowchart of the steps of a file processing method of the present application.

[0018] Figure 3 It is a schematic diagram of a file processing method of the present application.

[0019] Figure 4 It is a schematic diagram of a file processing method of the present application.

[0020] Figure 5It is a flowchart of the steps of a file processing method of the present application.

[0021] Figure 6 It is a flowchart of the steps of a file processing method of the present application.

[0022] Figure 7 It is a flowchart of the steps of a file processing method of the present application.

[0023] Figure 8 It is a schematic diagram of a file processing method of the present application.

[0024] Fig. 9 It is a structural block diagram of a file processing device of the present application.

[0025] Fig.10 It is a structural block diagram of a file processing device of the present application.

[0026] Fig.11 It is a block diagram of an electronic device of the present application.

[0027] Fig.12 It is a block diagram of an electronic device of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] In order to reduce the overhead of managing a large number of small files and improve the utilization of the storage space of the object storage system, the method of merging small files can be adopted. Multiple small files can be merged into a large file, and the merged large file can be stored in the object storage system. Then, the original small files can be deleted in the object storage system, and the normal indexing of small files can be ensured by recording the mapping relationship between the original small files and a certain piece of content in the merged large file. After the small files are merged into the large files, if the small files are deleted, invalid holes will be generated in the large files. Therefore, these holes need to be recycled to avoid a large amount of invalid space occupation. However, the efficiency of hole recycling will affect the load and disk loss of the entire object storage system.

[0030] To this end, the present solution of the present application is proposed.

[0031] Before introducing the solution of the present application, the technical terms that may be involved in the present application are first explained.

[0032] Object storage: Stores various types of data as "objects", each of which is identified by a unique identifier and can contain metadata and actual data.

[0033] Bucket: A logical concept in object storage, representing a container for storing objects.

[0034] Lifecycle: A feature in object storage that enables automatic deletion of files upon expiration. By setting a lifecycle for a bucket, eligible objects in the bucket will be automatically deleted after being stored for a specified period of time.

[0035] Small file merging: Aggregate several files that take up smaller space into one file that takes up larger space for saving.

[0036] Hole recovery: When a small file corresponding to a merged large file is deleted by the user, it is equivalent to creating a hole corresponding to the small file in the large file. When there are too many holes, hole recovery is required to avoid wasting storage space.

[0037] Reference Figure 1 , shows a flowchart of the steps of a file processing method of the present application, wherein the method comprises:

[0038] In step S101, a plurality of first files are screened in the file system, and the storage times when the plurality of first files are stored in the file system are the same.

[0039] There are a large number of files in the file system. This application does not limit the types of files in the file system. Whenever a file is stored in the file system, the binding relationship between the file identifier of the file and the storage time when the file is stored in the file system will be recorded in the file system. A large number of files in the file system can be stored in a high-performance cache pool to ensure read and write performance. The storage time can be divided into days, for example, divided into 24 hours, or divided into 48 hours, etc.

[0040] In this step, a plurality of first files are screened in the file system according to a binding relationship between a file identifier of a file recorded in the file system and a storage time when the file is stored in the file system.

[0041] The file system in this application may include an object storage system.

[0042] In step S102, the deletion time at which each of the plurality of first files is expected to be deleted from the file system is obtained.

[0043] In one embodiment of the present application, for any first file among multiple first files, the deletion time when the first file is expected to be deleted from the file system can be obtained through the following process, and the same is true for each other first file among the multiple first files.

[0044] The specific process includes:

[0045] 1021. Obtain the life cycle of the first file in the file system.

[0046] The life cycle of a file in the file system can be understood as: the length of time that a file will persist in the file system. If this duration is exceeded, the file will be automatically deleted from the file system.

[0047] This step can be achieved through the following process, including:

[0048] 11) Detect whether the file system has file attributes of the first file, where the file attributes of the file include at least one of the following: the type of the file, the prefix or suffix of the file name, the file bucket to which the file belongs, and the user to whom the file belongs.

[0049] 12) When the file system has the file attributes of the first file, the life cycle of the first file in the file system is obtained according to the file attributes of the first file.

[0050] In one embodiment of the present application, the file attributes of the first file include the file bucket to which the first file belongs. In this way, when obtaining the life cycle of the first file in the file system according to the file attributes of the first file, it can be determined whether the life cycle of the file bucket to which the first file belongs has been set in the file system. In the case that the life cycle of the file bucket to which the first file belongs has been set in the file system, the life cycle of the first file in the file system is obtained according to the life cycle of the file bucket to which the first file belongs. For example, the life cycle of the file bucket to which the first file belongs is determined as the life cycle of the first file in the file system. The life cycle of the file bucket is set in advance by the technician based on the actual situation of the files in the file bucket. For example, the basic deletion period of files in the file bucket named one-week-data is 7 days. In this way, the life cycle of the file bucket named one-week-data can be set to 7 days.

[0051] The file system has a gateway, and the file system can be accessed through the gateway of the file system. In this way, the life cycle of the file bucket to which the first file belongs can be obtained through the gateway of the file system.

[0052] Alternatively, in another embodiment of the present application, the file attribute of the first file includes a prefix or suffix of the file name of the first file. In this way, when obtaining the life cycle of the first file in the file system according to the file attribute of the first file, the front and back or suffix of the file name of the first file can be obtained. In the correspondence between the prefix or suffix of the file name and the life cycle of the file in the file system, the life cycle corresponding to the front and back or suffix of the file name of the first file is searched and used as the life cycle of the first file in the file system.

[0053] The correspondence between the prefix or suffix of the file name and the life cycle of the file in the file system is set in advance by the technician. For example, most files starting with or ending with tmp have a deletion period of 1 day. Thus, in the correspondence between the prefix or suffix of the file name and the life cycle of the file in the file system, the life cycle corresponding to the prefix or suffix of tmp is 1 day.

[0054] 13) When the file system does not have the file attribute of the first file, the life cycle of the first file in the file system is set to a specific duration.

[0055] When the file system does not have the file attribute of the first file, it indicates that the first file has no relevant life cycle configuration.

[0056] In one embodiment, the specific duration may be 3 months, 4 months, 5 months, 6 months, 9 months, 12 months, 18 months or 24 months, etc., and may be determined based on actual conditions, and this application does not impose any limitation on this.

[0057] Small files without a life cycle are likely to be retained in the file system for a long time, and the merged large files will hardly have holes in a short time. Therefore, a longer life cycle can be set for the merged large files, so that the merged large files will not be processed in a short time, avoiding invalid scanning.

[0058] 1022. Obtain a deletion time when the first file is expected to be deleted from the file system according to a storage time when the first file is stored in the file system and a life cycle of the first file in the file system.

[0059] For example, the sum of the storage time when the first file is stored in the file system and the life cycle of the first file in the file system is calculated to obtain the deletion time when the first file is expected to be deleted from the file system.

[0060] In step S103, a plurality of second files are screened from the plurality of first files, and the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold.

[0061] In one embodiment, the plurality of first files are sorted in order from earliest to latest according to the deletion time when each of the first files is expected to be deleted from the file system.

[0062] Starting from the first file ranked first among the multiple first files, the first files are selected in sequence until the total space occupied by the selected multiple first files is close to or equal to the preset maximum space of the large file. The preset maximum space of the large file can be 128MB, 256MB or 512MB, etc., which can be determined according to actual conditions, and this application is not limited to this.

[0063] A plurality of first files are selected as second files, and the process of step S104 to step S107 is executed on the second files.

[0064] If there are still remaining first files, starting from the first file ranked first among the remaining first files, first files are selected in sequence until the total space occupied by the selected multiple first files is close to or equal to the preset maximum space of the large file.

[0065] A plurality of first files are selected as second files, and the process of step S104 to step S107 is executed on the second files.

[0066] This continues until the process of step S104 to step S107 is completed for all first files.

[0067] Alternatively, in another embodiment, the plurality of first files are sorted in order from earliest to latest according to the deletion time when each of the first files is expected to be deleted from the file system.

[0068] Starting from the first file ranked first among the multiple first files, N adjacent second files are sequentially screened in the multiple first files in the sorted order, where N is greater than or equal to 2. The total space occupied by the N second files is less than or equal to the preset space, and the difference between the deletion times when the N second files are respectively expected to be deleted from the file system is less than a first preset threshold.

[0069] The process of step S104 to step S107 is executed for the screened N second files.

[0070] If there are still remaining first files (for example, the number of the plurality of first files is greater than N), starting from the first file ranked first among the remaining first files, M adjacent second files are sequentially screened in the remaining first files in the sorted order, where M is greater than or equal to 2. The total space occupied by the M second files is less than or equal to the preset space, and the difference between the deletion times when the M second files are respectively expected to be deleted from the file system is less than a first preset threshold.

[0071] The process of step S104 to step S107 is executed on the screened M second files.

[0072] This continues until the process of step S104 to step S107 is completed for all first files.

[0073] The first preset threshold value can be 1 day, 2 days or 3 days, etc., which can be determined according to actual conditions and is not limited in this application.

[0074] In step S104, multiple second files are merged into a third file.

[0075] In the present application, the specific method of merging multiple second files into a third file can refer to the currently existing method, and the present application does not limit this.

[0076] In step S105, the third file is stored in the file system.

[0077] The third file is a large file, so the third file can be stored in a common storage space in the file system, rather than in a high-performance cache pool in the file system, to avoid excessive occupation of the high-performance cache pool.

[0078] In step S106, a first correspondence between the identifiers of each second file, the identifier of the third file, and the offset addresses of each second file in the third file is stored in the file system.

[0079] For example, for any second file among the plurality of second files, the identifier of the second file, the identifier of the third file, and the offset address of the second file in the third file may be combined into a corresponding table entry to obtain the corresponding table entry corresponding to the second file, and the same is true for each of the other second files among the plurality of second files, thereby obtaining corresponding table entries corresponding to the plurality of second files, and combining the corresponding table entries corresponding to the plurality of second files to obtain the first corresponding relationship. This facilitates the subsequent indexing of a second file according to the first corresponding relationship.

[0080] In the present application, after executing step S104, step S105 and step S106 may be executed in parallel or one after the other, and the present application does not impose any limitation on this.

[0081] After step S105 and step S106 are executed, step S107 may be executed.

[0082] In step S107, a plurality of second files are deleted in the file system.

[0083] Since the multiple second files have been merged into the third file, the second files previously stored in the file system are equivalent to duplicate storage. Therefore, in order to save storage space, the multiple second files can be deleted in the file system.

[0084] For example, the multiple second files are located in a high-performance cache pool in the file system, so the multiple second files can be deleted from the high-performance cache pool in the file system.

[0085] In the present application, a plurality of first files are screened in a file system, and the storage times of the plurality of first files when they are respectively stored in the file system are the same; the deletion times at which the plurality of first files are respectively expected to be deleted from the file system are obtained; a plurality of second files are screened in the plurality of first files, and the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold; the plurality of second files are merged into a third file; the third file is stored in the file system; a first correspondence between the identifiers of the respective second files, the identifiers of the third file, and the offset addresses of the respective second files in the third file is stored in the file system; and the plurality of second files are deleted in the file system.

[0086] In this way, the hole rate of the third file in the file system can be detected; when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; and the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file are deleted in the file system.

[0087] Since the difference between the deletion times of the multiple second files in the third file when they are expected to be deleted from the file system is less than the first preset threshold, the remaining second files in the third file will be all deleted within a short period of time after a second file in the third file is deleted for the first time, so that the hole rate of the third file will reach 100%, so that the third file can be directly deleted from the file system, and there is no need to re-merge the third file (the second file no longer exists in the third file), or the number of times the merged large file needs to be re-merged is reduced, the disk reading and writing is reduced, the file system load and disk loss are reduced, and the hole recovery efficiency is improved.

[0088] Further, in another embodiment of the present application, see Figure 2 , the method further comprises:

[0089] In step S201, the time when the third file is completely empty is obtained according to the deletion time when each of the plurality of second files is expected to be deleted from the file system.

[0090] For example, when the deletion times at which multiple second files are expected to be deleted from the file system are exactly the same, the time when the third file is completely empty is obtained based on the deletion time at which any one of the multiple second files is expected to be deleted from the file system. For example, the deletion time at which any one of the multiple second files is expected to be deleted from the file system is used as the time when the third file is completely empty, so that when the time reaches the time when the third file is completely empty, the emptiness rate of the third file can reach 100%.

[0091] Or, for another example, when the deletion times at which multiple second files are expected to be deleted from the file system are not exactly the same, among the deletion times at which the multiple second files are expected to be deleted from the file system, the latest deletion time is selected, and the time when the third file is completely empty is obtained based on the latest deletion time. For example, the latest time is used as the time when the third file is completely empty, so that when the time reaches the time when the third file is completely empty, the emptiness rate of the third file can reach 100%.

[0092] In step S202, a second correspondence between the file identifier of the third file and the moment when the third file is completely empty is stored in the file system.

[0093] This facilitates the subsequent use of the second corresponding relationship between the file identifier of the third file and the moment when the third file is completely empty, for example, used in step S404 in subsequent embodiments.

[0094] In this application, different files have different file identifiers.

[0095] by Figure 3 and Figure 4 This solution is illustrated for the purpose of example, but is not intended to limit the scope of protection of this application. Figure 3 A schematic diagram of the merging of small files with lifecycles configured. Figure 4 A schematic diagram of the merging of small files without a configured lifecycle.

[0096] Reference Figure 5 , shows a flowchart of the steps of a file processing method of the present application, wherein the method comprises:

[0097] In step S301, the hole rate of a third file in the file system is detected, wherein the third file is obtained by merging a plurality of second files and stored in the file system, the plurality of second files are selected from a plurality of first files in the file system, the difference between the deletion times when the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold, and the storage times when the plurality of first files are respectively stored in the file system are the same.

[0098] For example, the third file may be stored in the file system through the process of step S101 to step S105 in the aforementioned embodiment.

[0099] In the present application, the third file is obtained by merging multiple second files, and some of the second files in the third file may be deleted from the third file. In this way, the hole rate of the third file can be understood as: the ratio of the number of second files that have been deleted in the third file to the total number of second files in the third file, and the total number of second files in the third file is: the number of second files in the third file when the third file is just merged.

[0100] In one embodiment, the hole rate of each third file in the file system may be detected periodically or in real time.

[0101] Alternatively, in another embodiment, the third files whose completely empty moments are earlier than or equal to the current moment are screened in the file system, and then the hole rates of the third files whose completely empty moments are earlier than or equal to the current moment in the file system are detected. Alternatively, in another embodiment, for any third file in the file system, it is possible to detect whether the third file is completely empty at a moment earlier than or equal to the current moment. If the third file is completely empty at a moment earlier than or equal to the current moment, the hole rate of the third file in the file system is detected again, and the same is true for each other third file in the file system. In this way, the hole rate detection of the third file is performed when the current moment reaches the moment when the third file is completely empty, which can avoid continuous invalid detection and reduce the load.

[0102] In step S302, when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system.

[0103] If the hollow rate of a third file is 100%, it means that the second files in the third file have been deleted from the third file, and the third file is an empty file. In this case, in order to save storage space, the third file can be deleted in the file system. The merge operation does not need to be performed again for the target third file.

[0104] In step S303, the first correspondence between the identifiers of the second files in the target third file, the identifier of the target third file, and the offset addresses of the second files in the target third file is deleted in the file system. The first correspondence is stored in the file system after the target third file is stored in the file system in advance.

[0105] After the target third file is deleted in the file system, the first correspondence between the identifiers of the respective second files in the target third file, the identifier of the target third file, and the offset addresses of the respective second files in the target third file previously stored in the file system is no longer useful. In order to save storage space, the first correspondence between the identifiers of the respective second files in the target third file, the identifier of the target third file, and the offset addresses of the respective second files in the target third file may be deleted in the file system.

[0106] The first corresponding relationship may be stored in the file system in step S106 in the aforementioned embodiment.

[0107] The target third file with a 100% hole rate can be directly deleted, which greatly reduces the number of overlaps and merges, reduces disk reading and writing, and improves the efficiency of hole recovery.

[0108] In the present application, the hole rate of a third file in a file system is detected; when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; and the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file in the target third file is deleted in the file system.

[0109] Since the difference between the deletion times of the multiple second files in the third file when they are expected to be deleted from the file system is less than the first preset threshold, the remaining second files in the third file will be all deleted within a short period of time after a second file in the third file is deleted for the first time, so that the hole rate of the third file will reach 100%, so that the third file can be directly deleted from the file system, and there is no need to re-merge the third file (the second file no longer exists in the third file), or the number of times the merged large file needs to be re-merged is reduced, the disk reading and writing is reduced, the file system load and disk loss are reduced, and the hole recovery efficiency is improved.

[0110] In the present application, there may be a third file whose hole rate does not reach 100% in the file system. The third file whose hole rate does not reach 100% can be regarded as the remaining third file, and it can be processed according to the following process, for example, see Figure 6 , the process includes:

[0111] In step S401, a plurality of candidate third files whose hole rates exceed a preset hole rate threshold are screened from the remaining third files.

[0112] The preset void rate threshold may be set by a technician according to actual conditions, for example, may include 50%, 55%, 60%, 65% or 70%, etc. This application does not limit the specific value.

[0113] If the hole rate of a third file exceeds a preset hole rate threshold, it often means that many second files in the third file have been deleted, the hole rate of the third file is high, and the remaining second files in the third file need to be re-merged. The remaining second files in the third file can be re-merged with the second files in other third files to release the space previously occupied by the deleted second files in the third file.

[0114] In step S402, the completely empty moments of each candidate third file are obtained.

[0115] In one embodiment, the moment when each candidate third file is completely empty can be queried according to the file identifier of each candidate third file and based on the second correspondence between the file identifier of the third file stored in the file system in step S202 and the moment when the third file is completely empty.

[0116] Alternatively, in one embodiment, if the second correspondence between the file identifier of the third file and the moment when the third file is completely empty has not been stored in the file system before, for any candidate third file among the candidate third files, the moment when the candidate third file is completely empty can be obtained in any one of the following two ways.

[0117] For example, when the remaining second files in the candidate third file are each expected to be deleted from the file system at exactly the same time, the time when the candidate third file is completely empty is obtained based on the time when any remaining second file in the candidate third file is expected to be deleted from the file system. For example, the time when any remaining second file in the candidate third file is expected to be deleted from the file system is used as the time when the candidate third file is completely empty.

[0118] Or, for another example, when the deletion times at which the remaining second files in the candidate third file are each expected to be deleted from the file system are not exactly the same, the latest deletion time is selected among the deletion times at which the remaining second files in the candidate third file are each expected to be deleted from the file system, and the time when the candidate third file is completely empty is obtained based on the latest deletion time, for example, the latest time is used as the time when the candidate third file is completely empty.

[0119] In step S403, multiple third files to be merged are selected from multiple candidate third files in the order of the complete hole moments of each candidate third file from early to late and the hole rates from high to low when the complete hole moments are the same. The difference between the complete hole moments of the multiple third files to be merged is less than the second preset threshold.

[0120] In one embodiment, the candidate third files are sorted in order from earliest to latest according to the deletion time when they are expected to be deleted from the file system (if the deletion time is the same, they are sorted in order from high to low according to the hole rate).

[0121] Starting from the candidate third file that is ranked first among the candidate third files, the candidate third files are selected in sequence until the total space occupied by the selected multiple candidate third files is close to or equal to the preset maximum space of the large file. The preset maximum space of the large file can be 128MB, 256MB or 512MB, etc., which can be determined according to actual conditions, and this application is not limited to this.

[0122] A plurality of candidate third files are selected as the third files to be merged, and the process of step S404 to step S408 is executed on the third files to be merged.

[0123] If there are any remaining candidate third files, starting from the candidate third file ranked first among the remaining candidate third files, candidate third files are selected in sequence until the total space occupied by the selected multiple candidate third files is close to or equal to the preset maximum space of the large file.

[0124] A plurality of candidate third files are selected as the third files to be merged, and the process of step S404 to step S408 is executed on the third files to be merged.

[0125] This continues until the process from step S404 to step S408 is completed for all candidate third files.

[0126] The second preset threshold may be 1 day, 2 days or 3 days, etc., which may be determined according to actual conditions, and is not limited in this application. The second preset threshold may be the same as or different from the first preset threshold.

[0127] In step S404, the second files among the plurality of third files to be merged are merged into a fourth file.

[0128] In the present application, the specific method of merging the second file among multiple third files to be merged into the fourth file can refer to the currently existing method, and the present application does not limit this.

[0129] Through this embodiment, the deletion time of each small file in the fourth file which is a large file is relatively close. When the deletion time of the fourth file is reached, it is very likely that the hole rate of the fourth file has reached 100%, so the number of overlapping merging times can be greatly reduced.

[0130] In step S405, the fourth file is stored in the file system.

[0131] In step S406, a third correspondence between the identifiers of each second file in each third file to be merged, the identifier of the fourth file, and the offset addresses of each second file in each third file to be merged in the fourth file is stored in the file system.

[0132] For example, for any second file in any third file to be merged, the identifier of the second file, the identifier of the fourth file and the offset address of the second file in the fourth file can be combined into a corresponding table entry to obtain the corresponding table entry corresponding to the second file. The same is true for each other second file in the third file to be merged, and for each second file in each other third file to be merged, so as to obtain the corresponding table entry corresponding to each second file in each third file to be merged. The corresponding table entries corresponding to each second file in each third file to be merged are combined to obtain a third corresponding relationship.

[0133] This facilitates that when a second file needs to be indexed later, the second file can be indexed according to the third corresponding relationship.

[0134] In the present application, after executing step S404, step S405 and step S406 may be executed in parallel or one after the other, and the present application does not impose any limitation on this.

[0135] After step S405 and step S406 are executed, step S407 may be executed.

[0136] In step S407, each third file to be merged is deleted in the file system.

[0137] Since the second files in each of the third files to be merged have been merged into the fourth file, the third files to be merged previously stored in the file system are equivalent to duplicate storage. Therefore, in order to save storage space, each of the third files to be merged can be deleted in the file system.

[0138] In step S408, for any third file to be merged, the first correspondence between the identifiers of each second file in the third file to be merged, the identifier of the third file to be merged, and the offset addresses of each second file in the third file to be merged in the third file to be merged is deleted in the file system.

[0139] Among them, the first correspondence between the identifiers of each second file in the third file to be merged, the identifier of the third file to be merged, and the offset addresses of each second file in the third file to be merged in the third file to be merged is stored in the file system after the third file to be merged is stored in the file system in advance.

[0140] After deleting each third file to be merged in the file system, the first correspondence between the identifiers of each second file in the third file to be merged, the identifiers of the third file to be merged, and the offset addresses of each second file in the third file to be merged, which were previously stored in the file system, is no longer useful. Therefore, in order to save storage space, the first correspondence between the identifiers of each second file in the third file to be merged, the identifiers of the third file to be merged, and the offset addresses of each second file in the third file to be merged, can be deleted in the file system.

[0141] In addition, for the third file that has reached its completely empty moment but its void rate has not yet reached the preset void rate threshold, it indicates that its completely empty moment is too early, and its completely empty moment can be postponed to reduce invalid detections. For example, its completely empty moment can be postponed by 1 day, 2 days, 3 days or 7 days, etc., which can be determined according to actual conditions.

[0142] When the moment when the processed third file is completely empty is greater than the current moment, the hole rates of other third files that follow it are often lower than the preset hole rate threshold. Therefore, the hole recovery process for other third files that follow it can be suspended until the current moment reaches the moment when it is completely empty, thereby avoiding invalid detection.

[0143] Further, in another embodiment of the present application, see Figure 7 , the method further comprises:

[0144] In step S501, the time when the fourth file is completely empty is obtained according to the time when multiple third files to be merged are completely empty.

[0145] In one embodiment, if the moments when multiple third files to be merged are completely empty have been obtained, then in one embodiment, when the moments when multiple third files to be merged are completely empty are exactly the same, the moment when the fourth file is completely empty can be obtained based on the moment when any one of the multiple third files to be merged is completely empty. For example, the moment when any one of the multiple third files to be merged is completely empty can be used as the moment when the fourth file is completely empty, so that when the time reaches the moment when the fourth file is completely empty, the empty rate of the fourth file can reach 100%.

[0146] Alternatively, in another embodiment, when the moments when multiple third files to be merged are completely empty are not exactly the same, the latest moment when the third files to be merged are completely empty is selected from the moments when the third files to be merged are completely empty, and the moment when the fourth file is completely empty is obtained based on the latest moment when the third file is completely empty. For example, the latest moment when the fourth file is completely empty is used as the moment when the fourth file is completely empty, so that when the time reaches the moment when the third file is completely empty, the hole rate of the third file can reach 100%.

[0147] In step S502, a fourth correspondence between the file identifier of the fourth file and the moment when the fourth file is completely empty is stored in the file system.

[0148] When it is necessary to obtain the moment when the fourth file is completely empty later, the fourth corresponding relationship between the file identifier of the fourth file and the moment when the fourth file is completely empty can be used to obtain the moment when the fourth file is completely empty.

[0149] In step S503, for any third file to be merged, in the file system, the second corresponding relationship between the file identifier of the third file to be merged and the moment when the third file to be merged is completely empty is deleted.

[0150] After deleting each third file to be merged in the file system, the second correspondence between the file identifier of the third file previously stored in the file system and the moment when the third file is completely empty is no longer useful. Therefore, in order to save storage space, the second correspondence between the file identifier of the third file and the moment when the third file is completely empty can be deleted in the file system.

[0151] The second correspondence is stored in the file system after the third file to be merged is stored in the file system in advance. The moment when the third file to be merged in the second correspondence is completely empty is obtained according to the deletion moment when each second file in the third file to be merged is expected to be deleted from the file system. For example, it can be stored in the file system in step S202 in the aforementioned embodiment.

[0152] by Figure 8 This solution is illustrated for the purpose of example, but is not intended to limit the scope of protection of this application. Figure 8 Schematic diagram of void recovery.

[0153] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions involved are not necessarily required by the present application.

[0154] Reference Fig. 9 , shows a file processing device of the present application, the device comprising: a first screening module 11, used to screen multiple first files in a file system, the storage time of the multiple first files when they are respectively stored in the file system is the same; a first acquisition module 12, used to obtain the deletion time when the multiple first files are respectively expected to be deleted from the file system; a second screening module 13, used to screen multiple second files from the multiple first files, the difference between the deletion time when the multiple second files are respectively expected to be deleted from the file system is less than a first preset threshold; a first merging module 14, used to merge the multiple second files into a third file; a first storage module 15, used to store the third file in the file system; a second storage module 16, used to store in the file system a first correspondence between the identifier of each second file, the identifier of the third file and the offset address of each second file in the third file; a first deleting module 17, used to delete the multiple second files in the file system.

[0155] The device also includes: a second acquisition module, used to obtain the moment when the third file is completely empty based on the deletion moments when each of the multiple second files is expected to be deleted from the file system; and a third storage module, used to store in the file system a second correspondence between the file identifier of the third file and the moment when the third file is completely empty.

[0156] The second acquisition module includes: a first acquisition unit, which is used to obtain the moment when the third file is completely empty according to the deletion moment when any one of the multiple second files is expected to be deleted from the file system when the deletion moments when the multiple second files are each expected to be deleted from the file system are exactly the same; or, a second acquisition unit, which is used to select the latest deletion moment among the deletion moments when the multiple second files are each expected to be deleted from the file system when the deletion moments when the multiple second files are each expected to be deleted from the file system are not exactly the same, and obtain the moment when the third file is completely empty according to the latest deletion moment.

[0157] In the present application, a plurality of first files are screened in a file system, and the storage times of the plurality of first files when they are respectively stored in the file system are the same; the deletion times at which the plurality of first files are respectively expected to be deleted from the file system are obtained; a plurality of second files are screened in the plurality of first files, and the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold; the plurality of second files are merged into a third file; the third file is stored in the file system; a first correspondence between the identifiers of the respective second files, the identifiers of the third file, and the offset addresses of the respective second files in the third file is stored in the file system; and the plurality of second files are deleted in the file system.

[0158] In this way, the hole rate of the third file in the file system can be detected; when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; and the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file are deleted in the file system.

[0159] Since the difference between the deletion times of the multiple second files in the third file when they are expected to be deleted from the file system is less than the first preset threshold, the remaining second files in the third file will be all deleted within a short period of time after a second file in the third file is deleted for the first time, so that the hole rate of the third file will reach 100%, so that the third file can be directly deleted from the file system, and there is no need to re-merge the third file (the second file no longer exists in the third file), or the number of times the merged large file needs to be re-merged is reduced, the disk reading and writing is reduced, the file system load and disk loss are reduced, and the hole recovery efficiency is improved.

[0160] Reference Fig.10, shows a file processing device of the present application, the device comprising: a detection module 21, used to detect the hole rate of a third file in a file system; wherein the third file is obtained by merging a plurality of second files and stored in the file system, the plurality of second files are screened from a plurality of first files in the file system, the difference between the deletion times when the plurality of second files are expected to be deleted from the file system is less than a first preset threshold, and the storage times when the plurality of first files are respectively stored in the file system are the same; a second deletion module 22, used to delete the target third file in the file system when a target third file with a hole rate of 100% is detected in the file system; a third deletion module 23, used to delete the first correspondence between the identifiers of the respective second files in the target third file, the identifier of the target third file, and the offset addresses of the respective second files in the target third file in the target third file in the file system; wherein the first correspondence is stored in the file system after the target third file is stored in the file system in advance.

[0161] The device further includes: a third screening module, used to screen multiple candidate third files whose hole rates exceed a preset hole rate threshold value from the remaining third files; a third acquisition module, used to acquire the time when each candidate third file is completely empty; a fourth screening module, used to screen multiple third files to be merged from the multiple candidate third files in the order of the time when each candidate third file is completely empty from early to late and in the order of the hole rates from high to low when the time when each candidate third file is completely empty is the same, wherein the difference between the time when each of the multiple third files to be merged is less than a second preset threshold; a second merging module, used to merge the second files among the multiple third files to be merged into a fourth file; a fourth storage module, used to store the fourth file in the file system; and a fifth storage module, used to store the identifiers of each second file, the identifiers of the fourth file, and each of the third files to be merged in the file system. a third correspondence between the offset addresses of the second files in the third file to be merged and the fourth file respectively; a fourth deletion module, used to delete the third files to be merged in the file system; a fifth deletion module, used to delete, for any third file to be merged, the first correspondence between the identifiers of the second files in the third file to be merged, the identifier of the third file to be merged and the offset addresses of the second files in the third file to be merged in the third file to be merged in the file system; wherein the first correspondence between the identifiers of the second files in the third file to be merged, the identifier of the third file to be merged and the offset addresses of the second files in the third file to be merged in the third file to be merged is stored in the file system after the third file to be merged is stored in the file system in advance.

[0162] The device further includes: a fourth acquisition module, used to acquire the moment when the fourth file is completely empty based on the moments when multiple third files to be merged are completely empty; a sixth storage module, used to store, in the file system, a fourth correspondence between the file identifier of the fourth file and the moment when the fourth file is completely empty; a sixth deletion module, used to delete, in the file system, the second correspondence between the file identifier of the third file to be merged and the moment when the third file to be merged is completely empty for any third file to be merged; the second correspondence is stored in the file system after the third file to be merged is stored in the file system in advance, and the moment when the third file to be merged is completely empty in the second correspondence is acquired based on the deletion moment when each second file in the third file to be merged is expected to be deleted from the file system.

[0163] In the present application, the hole rate of a third file in a file system is detected; when a target third file with a hole rate of 100% is detected in the file system, the target third file is deleted in the file system; and the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file in the target third file is deleted in the file system.

[0164] Since the difference between the deletion times of the multiple second files in the third file when they are expected to be deleted from the file system is less than the first preset threshold, the remaining second files in the third file will be all deleted within a short period of time after a second file in the third file is deleted for the first time, so that the hole rate of the third file will reach 100%, so that the third file can be directly deleted from the file system, and there is no need to re-merge the third file (the second file no longer exists in the third file), or the number of times the merged large file needs to be re-merged is reduced, the disk reading and writing is reduced, the file system load and disk loss are reduced, and the hole recovery efficiency is improved.

[0165] Optionally, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0166] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] Fig.11 800 is a block diagram of an electronic device 800 shown in the present application. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0168] Reference Fig.11 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0169] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0170] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, images, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0171] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0172] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also monitor the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0173] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0174] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0175] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can monitor the open / closed state of the device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also monitor the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to monitor the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0176] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast operation information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (I rDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0177] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0178] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by a processor 820 of an electronic device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] Fig.121 is a block diagram of an electronic device 1900 shown in the present application. For example, the electronic device 1900 may be provided as a server.

[0180] Reference Fig.12 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0181] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™ or the like.

[0182] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0183] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0184] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0185] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0186] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0187] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0188] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0189] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0190] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0191] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A file processing method, characterized in that: The method comprises: Screening a plurality of first files in the file system, wherein the plurality of first files are stored at the same time when they are stored in the file system; Obtaining a deletion time at which each of the plurality of first files is expected to be deleted from the file system; Screening a plurality of second files from the plurality of first files, wherein the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold; Merging the plurality of second files into a third file; storing a third file in the file system; storing in the file system a first correspondence between the identifiers of each second file, the identifier of the third file, and the offset addresses of each second file in the third file; A plurality of second files are deleted in the file system.

2. The method according to claim 1, characterized in that The method further comprises: According to the deletion time when each of the plurality of second files is expected to be deleted from the file system, obtaining the time when the third file is completely empty; In the file system, a second corresponding relationship between the file identifier of the third file and the moment when the third file is completely empty is stored.

3. The method according to claim 2, characterized in that The step of obtaining the time when the third file is completely empty according to the deletion time when each of the plurality of second files is expected to be deleted from the file system comprises: When the deletion time at which the plurality of second files are respectively expected to be deleted from the file system is exactly the same, obtaining the time at which the third file is completely empty according to the deletion time at which any second file among the plurality of second files is expected to be deleted from the file system; or, When the deletion times at which multiple second files are expected to be deleted from the file system are not exactly the same, the latest deletion time is selected among the deletion times at which the multiple second files are expected to be deleted from the file system, and the time when the third file is completely empty is obtained based on the latest deletion time.

4. A file processing method, characterized in that: The method comprises: Detecting a hole rate of a third file in the file system; wherein the third file is obtained by merging a plurality of second files and stored in the file system, the plurality of second files are selected from a plurality of first files in the file system, a difference between deletion times when the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold, and the storage times when the plurality of first files are respectively stored in the file system are the same; When a target third file with a hole rate of 100% is detected in the file system, deleting the target third file in the file system; Delete in the file system the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file respectively in the target third file; wherein the first correspondence is stored in the file system after the target third file is stored in the file system in advance.

5. The method according to claim 4, characterized in that The method further comprises: Selecting a plurality of candidate third files whose void rates exceed a preset void rate threshold from the remaining third files; Get the completely empty moment of each candidate third file; Selecting a plurality of third files to be merged from the plurality of candidate third files in the order of the complete hole moments of the candidate third files from early to late and in the order of the hole rates from high to low when the complete hole moments are the same; the difference between the complete hole moments of the plurality of third files to be merged is less than a second preset threshold; Merging the second file of the plurality of third files to be merged into a fourth file; storing a fourth file in the file system; storing in the file system a third correspondence between the identifiers of each second file in each of the third files to be merged, the identifier of the fourth file, and the offset addresses of each second file in each of the third files to be merged in the fourth file; Deleting each third file to be merged in the file system; For any third file to be merged, deleting in the file system a first correspondence between identifiers of each second file in the third file to be merged, the identifier of the third file to be merged, and offset addresses of each second file in the third file to be merged in the third file to be merged; Among them, the first correspondence between the identifiers of each second file in the third file to be merged, the identifier of the third file to be merged, and the offset addresses of each second file in the third file to be merged in the third file to be merged is stored in the file system after the third file to be merged is stored in the file system in advance.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining a time point at which a fourth file is completely empty according to the time points at which the plurality of third files to be merged are completely empty; In the file system, a fourth corresponding relationship between the file identifier of the fourth file and the moment when the fourth file is completely empty is stored; For any third file to be merged, in the file system, deleting a second correspondence between a file identifier of the third file to be merged and a moment when the third file to be merged is completely empty; The second correspondence is stored in the file system after the third file to be merged is stored in the file system in advance, and the moment when the third file to be merged in the second correspondence is completely empty is obtained according to the deletion moment when each second file in the third file to be merged is expected to be deleted from the file system.

7. A file processing device, characterized in that: The device comprises: A first screening module, used for screening a plurality of first files in the file system, wherein the plurality of first files are stored at the same time when they are stored in the file system; A first acquisition module, used to acquire a deletion time at which each of the plurality of first files is expected to be deleted from the file system; A second screening module is used to screen a plurality of second files from the plurality of first files, wherein the difference between the deletion times at which the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold; A first merging module, used for merging a plurality of second files into a third file; A first storage module, used for storing a third file in the file system; A second storage module, used for storing in the file system a first correspondence between the identifier of each second file, the identifier of the third file, and the offset address of each second file in the third file; The first deleting module is used to delete multiple second files in the file system.

8. A file processing device, characterized in that: The device comprises: a detection module, configured to detect a hole rate of a third file in the file system; wherein the third file is obtained by merging a plurality of second files and stored in the file system, the plurality of second files are selected from a plurality of first files in the file system, the difference between the deletion times when the plurality of second files are respectively expected to be deleted from the file system is less than a first preset threshold, and the storage times when the plurality of first files are respectively stored in the file system are the same; A second deleting module is used to delete the target third file in the file system when a target third file with a hole rate of 100% is detected in the file system; The third deletion module is used to delete in the file system the first correspondence between the identifiers of each second file in the target third file, the identifier of the target third file, and the offset addresses of each second file in the target third file respectively in the target third file; wherein the first correspondence is stored in the file system after the target third file is stored in the file system in advance.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the file processing method according to any one of claims 1 to 6 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the file processing method according to any one of claims 1 to 6 is implemented.

Images