File management method and device of file storage system and medium

Abstract

The invention discloses a file management method and device of a file storage system and a medium, relates to the technical field of file management, is used for solving the file management problem of the file storage system, and provides the file management method of the file storage system aiming at the problems that massive small files are not beneficial to data reading and the like. After a temporary file correction request is received, all temporary files in a target directory are converted into official files. And then, carrying out merging processing on the converted official file. Specifically, the method provides a merging configuration strategy which comprises a file size upper limit value and a file size lower limit value so as to define the standard of'small files' needing to participate in merging. According to the file merging realized based on the method, the number of files can be greatly reduced, the situation that a single file is too large due to excessive merging is avoided, and the data reading and analysis efficiency of the client side of the file storage system is better guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of file management, and particularly to a file management method, device and medium for a file storage system. Background Art

[0002] Nowadays, with the continuous progress of the performance of hardware devices such as computers and servers, the amount of data involved in various data-related business scenarios is also increasing. One common data storage and management system is the file storage system, which is a system that stores and manages data in the form of files.

[0003] In a file storage system, whenever new data is written, it is generally temporarily stored in the form of a temporary file. After a certain time period, the temporary file is then converted into a formal file for data analysis. This method has a good performance in avoiding data loss, but it also generates a large number of small files (i.e., files with a size smaller than a certain standard), which is not conducive to data reading. Especially when the client application of the file storage system needs to hold the metadata information of the file, a large number of small files will occupy a large amount of memory of the client, thereby affecting the data analysis efficiency of the client application. In severe cases, it may even lead to memory exhaustion and application crashes.

[0004] Therefore, those skilled in the art now urgently need a file management method for a file storage system to solve a series of problems brought by a large number of small files to the file storage system. Summary of the Invention

[0005] The purpose of the present invention is to provide a file management method, device and medium for a file storage system to solve a series of problems brought by a large number of small files to the file storage system.

[0006] To solve the above technical problems, the present invention provides a file management method for a file storage system, including:

[0007] After receiving a request for converting a temporary file into a formal file, convert the temporary file in the target directory into a formal file;

[0008] According to a preset merging configuration policy, merge the formal files; wherein, the merging configuration policy includes: a file size configuration item; the file size configuration item includes: a file size upper limit value and a file size lower limit value; when the size of the formal file exceeds the file size upper limit value, the formal file does not participate in file merging; when the size of the merged result file exceeds the file size lower limit value, the merged result file does not participate in subsequent file merging;

[0009] Return the file merging result; wherein, the file merging result includes the metadata of all the official files and the merged result file in the target directory.

[0010] In a possible embodiment, the merging configuration policy further includes: a merging directory configuration item; the merging directory configuration item records the directory of files that can be merged;

[0011] According to the preset merging configuration policy, merging the official files includes:

[0012] Merging the official files in the file directory in the target directory that belongs to the directory of files that can be merged.

[0013] In a possible embodiment, the merging configuration policy further includes: a sub-directory configuration item; the sub-directory configuration item includes two states: open and closed;

[0014] According to the preset merging configuration policy, merging the official files includes:

[0015] If the sub-directory configuration item is in the open state, then merge the official files in the file directory in the target directory that belongs to the directory of files that can be merged and the sub-directories of the directory of files that can be merged.

[0016] In a possible embodiment, the merging configuration policy further includes: a prohibited merging file configuration item; the prohibited merging file configuration item includes a prohibited merging suffix name;

[0017] According to the preset merging configuration policy, merging the official files includes:

[0018] If the file suffix name of the official file belongs to the prohibited merging suffix name, then the official file does not participate in the file merging.

[0019] In a possible embodiment, the merging configuration policy further includes: a file naming rule configuration item; the file naming rule configuration item includes: a prefix name, a suffix name, a system time, and an incrementing serial number;

[0020] According to the preset merging configuration policy, merging the official files includes:

[0021] When a new merged result file needs to be created, the file name of the merged result file is: the prefix name + the system time at the current moment + the incrementing serial number + the suffix name.

[0022] In a possible embodiment, according to the preset merging configuration policy, merging the official files includes:

[0023] For any file directory that needs to be merged, obtain the last merged result file according to the file naming rule configuration item and the dictionary order;

[0024] If the merged result file is not obtained, an empty file is created as the merged result file according to the file naming rule configuration item;

[0025] If the merged result file is obtained, determining whether the file size of the merged result file exceeds the file size lower limit;

[0026] If it exceeds, an empty file is created as the merged result file according to the file naming rule configuration item;

[0027] If not, obtaining any of the formal files in the current file directory, and determining whether the file size of the obtained formal file is greater than the file size upper limit;

[0028] If yes, skip the formal file, obtain the next formal file, and return to the step of determining whether the file size of the obtained formal file is greater than the file size upper limit;

[0029] If not, then the formal is merged into the obtained merge result file;

[0030] Repeat the above steps until all the formal files in the current file directory whose sizes are smaller than the file size upper limit are merged, and then proceed to merge files in the next file directory.

[0031] In a possible embodiment, converting the temporary files in the target directory into formal files includes:

[0032] Parsing the temporary file regularization request to determine the target directory;

[0033] By renaming the command, the temporary suffix in the file name of the temporary file in the target directory is deleted;

[0034] The temporary suffix is ​​a file suffix that indicates that the file is a temporary file.

[0035] In order to solve the above technical problems, the present invention also provides a file management device of a file storage system, comprising:

[0036] The regularization module is used to convert the temporary files in the target directory into formal files after receiving the request for regularization of temporary files;

[0037] A merging module, configured to merge the official files according to a preset merging configuration policy; wherein, the merging configuration policy includes: a file size configuration item; the file size configuration item includes: a file size upper limit value and a file size lower limit value; when the size of the official file exceeds the file size upper limit value, the official file does not participate in file merging; when the size of the merged result file exceeds the file size lower limit value, the merged result file does not participate in subsequent file merging;

[0038] A return module, configured to return a file merging result; wherein, the file merging result includes metadata of all the official files and the merged result file under the target directory.

[0039] To solve the above technical problems, the present invention further provides a file management device for a file storage system, including:

[0040] A memory, configured to store a computer program;

[0041] A processor, configured to implement the steps of the file management method of the file storage system as described above when executing the computer program.

[0042] To solve the above technical problems, the present invention further provides a non-volatile storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the file management method of the file storage system as described above are implemented.

[0043] A file management method for a file storage system provided by the present invention, after receiving a request for converting a temporary file into an official file, first converts all the temporary files under the target directory into official files for data analysis use. Further, since there may be many "small files" in the temporary files, after converting the temporary files into official files, the method further performs a merging process on the converted official files. Specifically, the method provides a merging configuration policy, including a file size configuration item. The file size configuration item includes a file size upper limit value and a file size lower limit value, and these two values are the criteria for defining what kind of file is a "small file" that needs to participate in merging. The file size upper limit value corresponds to the official file to be merged. When the file size of the official file is greater than the file upper limit value, it is considered that the official file is not a "small file" and does not need to participate in merging. On the contrary, an official file that does not meet the above conditions is a "small file" and needs to participate in merging. The file size lower limit value corresponds to the merged result file after merging. When the size of the merged result file is greater than the file size lower limit value, it means that the merged result file has merged enough "small files", and the merged result file is no longer regarded as a "small file", and other "small files" can be merged into a new merged result file.

[0044] In summary, based on the merge configuration strategy, the present method can merge a large number of small files in the file storage system, so that the file storage system only retains the official files that do not belong to the custom small file standard. Without losing data, this method greatly reduces the number of files in the file storage system, which is beneficial to data reading. In addition, file merging also reduces the number of file metadata that needs to be maintained in the file storage system, which is beneficial to reducing the resource occupation of the client memory and improving the data analysis efficiency of the client application.

[0045] The file management device of the file storage system and the non-volatile storage medium provided by the present invention correspond to the above method and have the same effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0047] Figure 1 It is a flowchart of a file management method for a file storage system provided by an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the physical architecture and process of file merging provided by an embodiment of the present invention;

[0049] Figure 3 It is a structural diagram of a file management device for a file storage system provided by an embodiment of the present invention;

[0050] Figure 4 It is a structural diagram of another file management device for a file storage system provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the protection scope of the present invention.

[0052] The core of the present invention is to provide a file management method, device and medium for a file storage system.

[0053] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0054] In the era of big data, business scenarios such as data analysis, advertising promotion, and model calculation have very high requirements for the timeliness of data. This also means that in related technologies, when using data storage and management systems such as file storage systems to store and manage data, it is generally necessary to convert different data written in real time into temporary files for storage, and after a certain period of time, convert the temporary files into formal files for data analysis.

[0055] Since the amount of data written at different times varies, temporary files in the file storage system may contain a large number of small files (i.e. files whose file size is smaller than a certain standard) over time. These large numbers of small files do not contain much data, but they contain a large amount of metadata that needs to be maintained. This is not conducive to data reading and will also occupy a large amount of memory resources to maintain and manage metadata, seriously affecting the data analysis efficiency of client applications. In severe cases, it may even cause memory exhaustion and application crashes.

[0056] To solve the above problems, the present invention provides a file management method for a file storage system, such as Figure 1 As shown, including:

[0057] S11: When a request for converting a temporary file into a formal file is received, the temporary file in the target directory is converted into a formal file.

[0058] S12: Merge the formal files according to the preset merge configuration strategy.

[0059] The merge configuration strategy includes: file size configuration item.

[0060] The file size configuration items include: the upper limit of the file size and the lower limit of the file size.

[0061] When the size of the formal file exceeds the upper limit of the file size, the formal file will not participate in the file merge. When the size of the merged result file exceeds the lower limit of the file size, the merged result file will not participate in the subsequent file merge.

[0062] S13: Return the file merging result.

[0063] The file merging result includes metadata of all formal files and merged result files in the target directory.

[0064] It should be noted that this method is a file management method for a file storage system, so it can be implemented through the file storage system. Further, according to different services, the file storage system can be divided into two sides: the server side and the client side. The server side is used to provide file storage and management services, while the client side is used to respond to user instructions and send file read and write requests to the server side. Exemplarily, the above steps S11 to S13 can be implemented by the server side in the file storage system, and the temporary file regularization request in step S11 is the request sent by the client to the server side. That is, in this embodiment, the trigger condition of the above method is that the client sends a temporary file regularization request to the server side. However, it should be noted that this is only a possible implementation scheme, and other trigger conditions can also be set in other embodiments. Generally speaking, when it is necessary to convert a temporary file into a formal file, this method can be executed to reduce the number of files.

[0065] Regarding how to convert a temporary file into a formal file, this embodiment does not limit it, but provides a possible implementation scheme. The above step S11 specifically includes:

[0066] S111: Parse the temporary file regularization request to determine the target directory.

[0067] S112: Through the rename instruction, delete the temporary suffix in the file name of the temporary file under the target directory.

[0068] Among them, the temporary suffix is the file suffix name indicating that this file is a temporary file.

[0069] Regarding the above step S111, it should be noted that when the client sends a temporary file regularization request to the server side, it is not necessarily a request to convert all temporary files in the file storage system. It may only be to convert the temporary files in some file directories. At this time, the temporary file regularization request should be attached with the file directory targeted by this regularization request, that is, the above target directory.

[0070] After determining the target directory, the object targeted by this temporary file regularization and even subsequent file merging can be determined, that is, the temporary files under the target directory. For step S112, in a possible application scenario, the file name of the temporary file in the file storage system generally has a ".tmp" suffix name. At this time, deleting the ".tmp" suffix name is a feasible solution to convert the temporary file into a formal file. Further, the above scheme of deleting the suffix name can be implemented through the rename instructions supported by the file storage system, such as: mv and rename instructions. Both of these instructions can implement the function of renaming files, and can delete the temporary suffix in the file name of the temporary file, thus completing the regularization.

[0071] Based on the above, this embodiment provides a simple and feasible solution for converting temporary files into formal files, which can be achieved by using the rename instruction originally supported by the file storage system. The implementation is simple and does not rely on other devices.

[0072] After that, for step S12. Step S12 is also the core step of this method and is the step to achieve file merging. In step S12, the formal files in the target directory can be merged. It is easy to know that when merging files, generally the files in the same-named directory are merged, and the merged result file is also located in this file directory. In addition, based on the number and size of the files in the target directory, one target directory may merge into one or more merged result files. It is easy to understand that the purpose of file merging is to merge multiple small files into one large file (both small and large here are relative concepts, that is, when the file size meets a certain standard, it can be regarded as a small file or a large file here). However, in actual applications, it is not desirable for the merged file to be too large, as this will make it difficult to read the target data and is not conducive to improving the data reading efficiency.

[0073] Therefore, this invention proposes a merging configuration strategy, which includes a file size configuration item. The file size configuration item specifically includes two thresholds: the upper limit value of the file size and the lower limit value of the file size.

[0074] Among them, the upper limit value of the file size is used to limit the size of the formal file. That is, when the file size of the formal file is greater than the upper limit value of the file size, it is considered that this formal file does not belong to the above-mentioned "small file" and no merging process is required. In this way, on the one hand, unnecessary merging operations are saved, and on the other hand, it can also prevent the merged result file from being too large, resulting in the need to screen specific data from a file with a too large data volume when reading specific data.

[0075] The lower limit value of the file size, on the other hand, is for the merged result file and is used to limit the size of the merged result file. As described above, file merging is not simply to merge all the formal files in the same-named directory into one merged result file. Depending on the number and size of the formal files contained in the target directory, to avoid a merged result file being too large, multiple merged result files can be merged. The lower limit value of the file size is for this purpose. When performing file merging, if the file size of the merged result file is less than the lower limit value of the file size, it means that this merged result file still belongs to the above-mentioned "small file" and continues to participate in file merging, that is, other formal files are merged into this merged result file. When the file size of the merged result file is greater than the lower limit value of the file size, it means that this merged result file no longer belongs to the above-mentioned "small file" and there is no need to continue file merging to avoid a single file being too large.

[0076] It should also be noted that there is no size or other restrictive relationship between the above upper limit value and lower limit value of the file size. Essentially, both thresholds are criteria for defining "small files" in file merging, to clarify which files are "small files" that need to be merged and which files are not "small files" and do not need to be merged. It's just that the two thresholds target different objects. The upper limit value of the file size defines the file size of the official file (i.e., the file to be merged), and the lower limit value of the file size defines the file size of the merged result file (i.e., the file after merging). Therefore, the two thresholds can be custom-set according to actual needs when taking values. However, in a possible implementation, the two thresholds can be equal to define the same criterion for "small files" and are also easier to implement.

[0077] Regarding step S13, it is the step of returning the file merging result in step S12. As in the above embodiment, when this method is implemented through the server in the file storage system and the temporary file regularization request received in step S11 is sent from the client in the file storage system, then this method is a step triggered in response to the request sent by the client. At this time, the result returned in step S13 is also to return the result to the client to achieve the synchronization of metadata after file merging.

[0078] Furthermore, the file merging result returned in step S13 includes the metadata of all official files and merged result files in the target directory. That is to say, it informs the client of the metadata of all remaining files after file merging in the target directory this time. Based on this, this embodiment also provides a possible implementation. For the client side of the file storage system, this method further includes:

[0079] S20: When receiving the file merging result, cache the metadata of all files in the target directory in memory.

[0080] That is to say, in this embodiment, after the client obtains the metadata of all new files after file merging in the target directory, it caches this part of metadata in memory. Thus, when subsequently responding to requests such as file reading or writing by the user, the corresponding file metadata can be quickly retrieved from memory, thereby achieving the purpose of improving the data reading and analysis efficiency of the client application.

[0081] Furthermore, as Figure 2 shown, it is a schematic diagram of a file merging process. Among them, the client is divided into a write file client and a read file client based on different read and write operations. The write file client is the client that writes temporary files to the server. The read file client is also the client that receives the file merging result in this embodiment. After caching the metadata in the file merging result in memory, it can improve the data reading and analysis efficiency of the read file client. The overall process is as Figure 2As shown in the figure, the write file client writes a temporary file to the server. When the server receives a request to convert a temporary file to a formal file (regardless of whether the client sending the request to convert the temporary file to a formal file is the write file client or the read file client), the server converts the temporary file in the target directory into a formal file and performs file merging. After that, the result of the file merging is returned to the read file client, which contains the metadata of all files in the target directory after the file merging. The read file client caches this part of the metadata in memory for subsequent quick calls.

[0082] In addition, after multiple file mergings, based on the implementation provided in this embodiment, the metadata of all files after file merging in the server is in the memory of the client. As described above, it can greatly improve the efficiency of the client reading files from the server, thereby improving the data reading and analysis efficiency of the client application. At the same time, since this method can merge a large number of small files, even if the metadata of the files is returned to the client for caching in memory, the total amount of the metadata is relatively controllable, occupying less memory of the client, that is, having less impact on the client application and being more suitable for the file management needs in the actual file storage system.

[0083] In summary, for the file management method of a file storage system provided by the present invention, when converting a temporary file in the file storage system into a formal file, the formal file is merged to greatly reduce the number of files without losing data. Further, to ensure the effect of reducing the total number of files by file merging and to avoid excessive merging resulting in an overly large single file that affects the data reading and analysis efficiency, this embodiment also provides a merging configuration strategy. Specifically, it includes two thresholds, namely the upper limit value of the file size and the lower limit value of the file size, which are used to limit the objects and results of file merging, that is, the formal file and the merged result file, respectively, for realizing a custom-defined standard for defining "small files". When the file size of the formal file is greater than the upper limit value of the file size, it means that this formal file does not belong to a "small file" and does not need to be merged, avoiding an overly large single merged result file. When the file size of the formal file is less than or equal to the upper limit value of the file size, it means that this formal file belongs to a "small file" and needs to be merged to ensure the effect of reducing the number of files by this method. The same is true for the lower limit value of the file size, except that the object it targets is the merged result file, to ensure that the file size of the merged result file is not too small to affect the file merging effect and that the file size of the merged result file is not too large to cause the problem of an overly large single file. This method greatly reduces the number of files in the file storage system without losing data, which is beneficial for data reading. In addition, file merging also reduces the number of file metadata that needs to be maintained in the file storage system, which is beneficial for reducing the resource occupation of the client memory and for improving the data analysis efficiency of the client application.

[0084] On the other hand, the above file size configuration item is only one of the configuration items in the merging configuration strategy of this method. In actual applications, other configuration items can be added according to different needs to standardize the file merging process. Exemplarily, this embodiment also provides a possible implementation of the merging configuration strategy. The above merging configuration strategy further includes: a merged directory configuration item.

[0085] The merged directory configuration item records the directories of files that can be merged.

[0086] Then when performing the merging of official files according to the preset merging configuration strategy in step S12 above, it also complies with: merging the official files in the file directories that belong to the directories of files that can be merged in the target directory.

[0087] In this embodiment, the merging configuration strategy additionally includes a merged directory configuration item. The merged directory configuration item is used to specify the file directories in the file storage system where file merging can be performed. That is, when the client sends a request to convert a temporary file to an official file to the server, after the server finds the target directory, it also needs to determine whether each target directory belongs to the directories of files that can be merged specified in the merged directory configuration item. If it belongs, the subsequent file merging in step S12 is performed on this target directory. On the contrary, if it does not belong, only the temporary file is converted to an official file in this target directory, but the subsequent file merging in step S12 is not performed.

[0088] It is not difficult to understand that the merged directory configuration item in this embodiment further delimits the scope of the file directories for file merging in the server of the file storage system. That is, the scope of the file directories where file merging can be performed is artificially pre-configured, and the files therein can be merged. In other words, the files in the unconfigured file directories are files that are not allowed to be merged. Such a setting helps to avoid the merging of some important files in the file storage system.

[0089] Furthermore, as can be known from the above embodiments, file merging is mostly carried out in units of file directories. That is, when performing file merging, the files in the same-named directories are merged, and the files in the subdirectories under the file directory are not included. Based on this, when the client sends a request to convert a temporary file to an official file, its target directory should also be clearly defined for each target directory that needs to perform file merging based on this principle. However, it is easy to know that the file directory is a tree structure, and a file directory may contain many subdirectories, and each subdirectory may contain its own subdirectories. Therefore, when a certain file directory and all its subdirectories below need to perform file merging, listing each file directory separately will increase the complexity of the method implementation. Based on this, this embodiment provides a possible implementation. The above merging configuration strategy further includes: a subdirectory configuration item. The subdirectory configuration item includes two states: open and closed.

[0090] At this time, when the above-mentioned step S12 merges the official documents according to the preset merging configuration policy, it also follows that: if the sub-directory configuration item is in the open state, then for the target directory, the official documents in the file directories belonging to the mergeable file directory and the sub-directories of the mergeable file directory are merged.

[0091] It is not difficult to understand that if the sub-directory configuration item is in the closed state, file merging of the sub-directories of the target directory is not allowed.

[0092] Based on the above, this embodiment provides a configuration policy for whether file merging can be performed on sub-directories. This enables users to flexibly decide whether to include all sub-directories of a certain directory in the scope of file merging according to their different needs. This makes the file merging and subsequent management implemented based on this method simpler and easier to implement.

[0093] On the other hand, both of the above two embodiments limit the scope of file merging starting from the file directory. This embodiment also provides an implementation scheme to further limit the scope of file merging from another perspective. Specifically, the above-mentioned merging configuration policy further includes: a prohibited merge file configuration item. The prohibited merge file configuration item includes prohibited merge suffixes.

[0094] At this time, when the above-mentioned step S12 merges the official documents according to the preset merging configuration policy, it also follows that: if the file suffix of the official document belongs to the prohibited merge suffixes, then the official document does not participate in file merging.

[0095] That is to say, this embodiment provides a scope limitation for whether file merging can be performed in units of official documents. Specifically, this embodiment pre-configures prohibited merge suffixes that do not allow file merging in the merging configuration policy. That is, when an official document contains a suffix name that is the same as the prohibited merge suffixes, the official document is not allowed to be merged. The same as the above embodiment, the implementation scheme provided by this embodiment is beneficial to protecting important documents from being merged. The difference is that in the above embodiment, when protecting important documents from being merged, it is in units of file directories. And this embodiment is in units of file types, specifically protecting important documents of specific suffix names (file types) from being merged through the suffixes of the files. Combining the above embodiments can achieve a more comprehensive and three-dimensional important document protection mechanism to prevent important documents from being merged.

[0096] On the other hand, as can be seen from the above, when the present method performs file management, any number of target directories can be specified for converting temporary files into regular files and file merging. In the application of an actual file storage system, generally, the conversion of a large number of files and file merging are involved. Also, based on the file size configuration item provided in the above embodiment, it is possible to effectively avoid an overly large single merged result file. That is, in actual file merging, the number of output merged result files is relatively large. Based on this, the present embodiment provides a possible solution for naming merged result files. The above-mentioned merging configuration strategy further includes: a file naming rule configuration item. The file naming rule configuration item includes: a prefix name, a suffix name, the system time, and an incrementing serial number.

[0097] At this time, when the above-mentioned step S12 merges official files according to the preset merging configuration strategy, it also complies with: when a new merged result file needs to be created, the file name of the merged result file is: prefix name + the system time at the current moment + incrementing serial number + suffix name.

[0098] The present embodiment provides a naming rule for merged result files, that is, each merged result file obtained by merging is named in the form of prefix name + the system time at the current moment + incrementing serial number + suffix name, so as to distinguish different merged result files. Among them, including the system time at the current moment and the incrementing serial number in the file naming rule can distinguish the creation sequence between different merged result files. It is not difficult to understand that the system time at the current moment in the file naming rule refers to the system time when the merged result file is created, and is used to indicate the creation time of the merged result file. In addition, in the present embodiment, the incrementing serial number is also used to distinguish the creation sequence of merged result files. Whenever a merged result file is created, the incrementing serial number is incremented by one, so as to distinguish the creation sequence of different file merging results.

[0099] On the other hand, for the file merging step in the above-mentioned step S12, the present embodiment also provides a possible implementation solution for its specific implementation process. The above-mentioned step S12 specifically includes:

[0100] S121: For any file directory that needs to perform file merging, obtain the last merged result file according to the file naming rule configuration item and the lexicographical order.

[0101] S122: If no merged result file is obtained, create an empty file as the merged result file according to the file naming rule configuration item.

[0102] S123: If a merged result file is obtained, determine whether the file size of the merged result file exceeds the lower limit value of the file size.

[0103] S124: If it exceeds, create an empty file as the merged result file according to the file naming rule configuration item.

[0104] S125: If not exceeded, obtain any official file in the current file directory, and determine whether the file size of the obtained official file is greater than the upper limit value of the file size.

[0105] S126: If so, skip this official file, obtain the next official file, and return to the step of determining whether the file size of the obtained official file is greater than the upper limit value of the file size.

[0106] S127: If not, then merge this one officially into the obtained merged result file.

[0107] S128: Repeat the above steps S121 - S127 until all official files with file sizes less than the upper limit value in the current file directory are merged, and then proceed with the file merging of the next file directory.

[0108] As can be seen from the above, steps S121 - S127 are the file merging process for one file directory, and the file directories that need to be merged can be comprehensively determined through the target directory and the merging directory configuration item and sub - directory configuration item proposed in the above embodiments. This embodiment will not elaborate here.

[0109] For the obtaining of the last merged result file in step S121, its purpose is to achieve orderly file merging. From the above file size configuration item, it can be known that the present invention limits the size of the merged result file through the lower limit value of the file size. That is, when the merged result file is greater than the lower limit value of the file size, this merged result file no longer participates in subsequent file merging, and a new merged result file can be created. Therefore, step S121 needs to obtain the most recently created merged result file. The merged result file with an earlier creation time must have reached the lower limit value of the file size under the above logic and cannot participate in subsequent file merging.

[0110] And in the above step S121, when obtaining the last merged result file according to the file naming rule configuration item and the dictionary order, the file naming rule configuration item relied on can be the file naming rule configuration item provided in the above embodiments.

[0111] In addition, if the most recently created merged result file is not obtained, it means that for this file directory, file merging has just started, and at this time, there are no official files for file merging yet. The file merging process in this embodiment is to create an empty file as the merged result file, and continuously write the data of other official files into this merged result file to achieve file merging until the file size of this merged result file exceeds the lower limit value of the file size.

[0112] In addition, in step S125, when selecting official documents to be merged, they can be selected based on any logic, such as randomly or sorted according to the dictionary order of file names or other methods. It is easy to understand that when there is a configuration item for prohibited merged files based on the above embodiments, when selecting official documents to be merged, it is also necessary to consider whether this official document can participate in the merge.

[0113] Furthermore, this method performs file merging in units of file directories. When multiple different file directories need to be merged under the file storage system, multi-threaded parallelism can be used to simultaneously merge multiple file directories, thereby greatly improving the file merging efficiency.

[0114] In addition to the embodiments of the file management method of a file storage system provided in the above embodiments, the present invention also provides an embodiment corresponding to a computer program product. A computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the file management method of the file storage system described in any of the above embodiments can be implemented.

[0115] Since the embodiments of the computer program product part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the computer program product part, and will not be elaborated here for the time being.

[0116] In the above embodiments, the file management method of a file storage system is described in detail. The present invention also provides an embodiment corresponding to a file management device of a file storage system. It should be noted that the present invention describes the embodiments of the device part from two perspectives, one is from the perspective of functional modules, and the other is from the perspective of hardware.

[0117] From the perspective of functional modules, as Figure 3 shown, this embodiment provides a file management device for a file storage system, including:

[0118] A regularization module 11, configured to convert a temporary file in a target directory into an official file when receiving a temporary file regularization request;

[0119] A merging module 12, configured to merge the official files according to a preset merging configuration policy; wherein, the merging configuration policy includes: a file size configuration item; the file size configuration item includes: a file size upper limit value and a file size lower limit value; when the size of the official file exceeds the file size upper limit value, the official file does not participate in file merging; when the size of the merged result file exceeds the file size lower limit value, the merged result file does not participate in subsequent file merging;

[0120] A return module 13 for returning the file merging result; wherein, the file merging result includes the metadata of all the official files and the merged result file under the target directory.

[0121] Since the embodiments in the apparatus part correspond to those in the method part, please refer to the description of the embodiments in the method part for the embodiments in the apparatus part, which will not be elaborated here.

[0122] Figure 4 The structural diagram of a file management apparatus of a file storage system provided by another embodiment of the present invention is as Figure 4 shown. A file management apparatus of a file storage system includes: a memory 20 for storing computer programs;

[0123] A processor 21 for implementing the steps of the file management method of a file storage system as described in the above embodiment when executing the computer program.

[0124] The file management apparatus of a file storage system provided in this embodiment may include, but is not limited to, a mobile terminal, a personal computer, a workstation, etc.

[0125] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may further include an artificial intelligence (AI) processor, and the AI processor is used to process computational operations related to machine learning.

[0126] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201. After the computer program is loaded and executed by the processor 21, it can implement the relevant steps of a file management method of a file storage system disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be transient storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, a file management method of a file storage system, etc.

[0127] In some embodiments, a file management device of a file storage system may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0128] Those skilled in the art can understand that Figure 4 the structure shown in

[0129] does not constitute a limitation on a file management device of a file storage system, and may include more or fewer components than those shown in the figure.

[0130] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. A computer program is stored on the non-volatile storage medium, and when the computer program is executed by a processor, it implements the steps recorded in the foregoing method embodiments.

[0131] It can be understood that if the method in the above embodiments is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage media include: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs, etc., which can store program codes.

[0132] The above has introduced in detail a file management method, device, and medium of a file storage system provided by the present invention. Each embodiment in the specification is described in a progressive manner. The key point of each embodiment is the difference from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and reference can be made to the description in the method part for relevant parts. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

[0133] It should also be noted that in this specification, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article, or device including the said element.

Claims

1. A file management method for a file storage system, characterized in that, Including: When receiving a request to convert a temporary file to a formal file, convert the temporary file in the target directory to a formal file; Merge the formal files according to a preset merge configuration policy; wherein, the merge configuration policy includes: a file size configuration item; the file size configuration item includes: a file size upper limit value and a file size lower limit value; when the size of the formal file exceeds the file size upper limit value, the formal file does not participate in file merging; when the size of the merged result file exceeds the file size lower limit value, the merged result file does not participate in subsequent file merging; Return the file merge result; wherein, the file merge result includes the metadata of all the formal files and the merged result file in the target directory.

2. The file management method of the file storage system according to claim 1, characterized in that, The merge configuration policy further includes: a merge directory configuration item; the merge directory configuration item records the directory of files that can be merged; The step of merging the formal files according to the preset merge configuration policy includes: Merge the formal files in the file directory in the target directory that belongs to the directory of files that can be merged.

3. The file management method of the file storage system according to claim 2, characterized in that, The merge configuration policy further includes: a sub-directory configuration item; the sub-directory configuration item includes two states: open and closed; The step of merging the formal files according to the preset merge configuration policy includes: If the sub-directory configuration item is in the open state, merge the formal files in the file directory in the target directory that belongs to the directory of files that can be merged and the sub-directories of the directory of files that can be merged.

4. The file management method of the file storage system according to claim 1, wherein The merge configuration policy further includes: a prohibited merge file configuration item; the prohibited merge file configuration item includes prohibited merge suffix names; The step of merging the formal files according to the preset merge configuration policy includes: If the file suffix name of the formal file belongs to the prohibited merge suffix names, the formal file does not participate in file merging.

5. The file management method of the file storage system according to claim 1, characterized in that, The merge configuration policy further includes: a file naming rule configuration item; the file naming rule configuration item includes: a prefix name, a suffix name, a system time, and an incrementing serial number; The step of merging the formal files according to the preset merge configuration policy includes: When a new merged result file needs to be created, the file name of the merged result file is: the prefix name + the system time at the current moment + the incrementing serial number + the suffix name.

6. The file management method of the file storage system according to claim 5, characterized in that, The step of merging the formal files according to the preset merge configuration policy includes: For any file directory that needs to perform file merging, obtain the last merged result file according to the file naming rule configuration item and the lexicographical order; If the merged result file is not obtained, create an empty file as the merged result file according to the file naming rule configuration item; If the merged result file is obtained, determine whether the file size of the merged result file exceeds the file size lower limit value; If it exceeds, create an empty file as the merged result file according to the file naming rule configuration item; If it does not exceed, obtain any formal file in the current file directory and determine whether the file size of the obtained formal file is greater than the file size upper limit value; If so, skip the formal document, obtain the next formal document, and return to the step of determining whether the file size of the obtained formal document is greater than the upper limit value of the file size; If not, merge the formal document into the obtained merged result file; Repeat the above steps until all formal documents with file sizes less than the upper limit value of the file size in the current file directory are merged, and then proceed with the file merging of the next file directory.

7. The file management method of the file storage system according to any one of claims 1 to 6, characterized in that, The conversion of the temporary file in the target directory to a formal file includes: Parsing the conversion request of the temporary file to a formal file to determine the target directory; Deleting the temporary suffix in the file name of the temporary file in the target directory through a rename instruction; Wherein, the temporary suffix is a file suffix name indicating that this file is a temporary file.

8. A file management device for a file storage system, characterized in that, It includes: A conversion module, configured to convert the temporary file in the target directory to a formal file when receiving a conversion request of the temporary file to a formal file; A merging module, configured to merge the formal documents according to a preset merging configuration policy; wherein, the merging configuration policy includes: a file size configuration item; the file size configuration item includes: an upper limit value of the file size and a lower limit value of the file size; when the size of the formal document exceeds the upper limit value of the file size, the formal document does not participate in file merging; when the size of the merged result file exceeds the lower limit value of the file size, the merged result file does not participate in subsequent file merging; A return module, configured to return the file merging result; wherein, the file merging result includes the metadata of all the formal documents and the merged result file in the target directory.

9. A file management device for a file storage system, characterized in that, It includes: A memory, configured to store a computer program; A processor, configured to implement the steps of the file management method of the file storage system as described in any one of claims 1 to 7 when executing the computer program.

10. A non-volatile storage medium, characterized in that, A computer program is stored on the non-volatile storage medium, and when the computer program is executed by the processor, the steps of the file management method of the file storage system as described in any one of claims 1 to 7 are implemented.

Images