Metadata management method, system and equipment of file system and storage medium

Abstract

The invention discloses a metadata management method, system and device of a file system and a storage medium, and is applied to the technical field of storage, and the method comprises the steps that after a target file of the file system is opened, a write request for the target file is received; receiving attached parameter information corresponding to the write request based on a preset expansion interface; based on the attached parameter information, selecting an analysis module specified by the attached parameter information from each pre-input analysis module for generating predefined metadata; and writing the to-be-written data into the target file based on the write request, and analyzing the to-be-written data through the selected analysis module to generate metadata information of the target file. By applying the scheme provided by the invention, metadata management can be effectively realized, the flexibility of extended metadata is ensured, and the instantaneity of the metadata and the consistency of the metadata and file contents are ensured.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a metadata management method, system, device and storage medium for a file system. Background Art

[0002] Due to their high scalability and massive storage capacity, distributed file systems have become the preferred storage solution for unstructured data such as meteorological observation records and industrial sensor streams. Current unstructured data lacks a unified metadata framework and only supports basic metadata types (such as file name, size, timestamp, and other basic attributes). This presents drawbacks such as an inability to meet complex business needs, low content retrieval efficiency, and high management costs. For example, recording only basic attributes of meteorological data makes it difficult to support refined analysis. Metadata must be expanded to include customized attributes such as latitude and longitude coordinates, meteorological element type, and observation time accuracy to enable spatial indexing and multi-dimensional correlation analysis.

[0003] Furthermore, some current solutions utilize dedicated data management systems to manage data and extend data attributes. However, the metadata attribute definitions within the file system remain static, making expansion inflexible. That is, metadata fields are fixed. For example, if a user needs to add a "satellite sensor number" attribute to meteorological data, they would need to redesign the file system structure or rely on external database connections, which is complex and reduces storage and query efficiency.

[0004] In summary, how to effectively implement metadata management and ensure the flexibility of extending metadata is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] The present application provides a metadata management method, system, device and storage medium for a file system to effectively implement metadata management and ensure the flexibility of extending metadata.

[0006] To solve the above technical problems, this application provides the following technical solutions:

[0007] In a first aspect, the present application provides a metadata management method for a file system, comprising:

[0008] After opening a target file in the file system, receiving a write request for the target file;

[0009] Receiving accompanying parameter information corresponding to the write request based on a preset extended interface;

[0010] Based on the accompanying parameter information, selecting a parsing module specified by the accompanying parameter information from various pre-input parsing modules for generating predefined metadata;

[0011] Based on the write request, the data to be written is written into the target file, and the data to be written is parsed by the selected parsing module to generate metadata information of the target file.

[0012] In one embodiment, after generating the metadata information of the target file, the method further includes:

[0013] Storing the currently generated metadata information of the target file in a storage manner that does not overwrite the metadata information of the old version of the target file;

[0014] A version number is generated for the metadata information of the target file currently being generated, and the version number is filled into a metadata version record table corresponding to the target file.

[0015] In one embodiment, it further includes:

[0016] receiving a metadata version selection instruction for the target file;

[0017] retaining, in the metadata version record table of the target file, the version number specified by the metadata version selection instruction and deleting the remaining version numbers;

[0018] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0019] In one embodiment, it further includes:

[0020] When it is detected that the metadata version record table of the target file meets the set trigger condition, outputting automatic deletion verification information to a designated terminal;

[0021] When the total number of version numbers in the metadata version record table reaches a set first number threshold, and / or when the number of version numbers in the metadata version record table that have existed for a first time period reaches a set second threshold, the metadata version meets the trigger condition;

[0022] When a first feedback instruction indicating prohibition of automatic deletion is received, the automatic deletion process is terminated;

[0023] When a second feedback instruction indicating that automatic deletion is allowed is received, and / or when the first feedback instruction is not received within the waiting time, all version numbers of the target file that have existed in the metadata version record table for a period of time exceeding the first time period are deleted;

[0024] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0025] In one embodiment, it further includes:

[0026] A correspondence between a designated field in the metadata information of the target file and a file path of the target file is established, so that the content in the designated field serves as index data of the target file.

[0027] In one embodiment, it further includes:

[0028] Receiving the uploaded parsing module and listing the parsing module in a set of ineffective parsing modules;

[0029] Receive the parsing module validation instruction;

[0030] Moving the parsing module specified by the parsing module validation instruction from the ineffective parsing module set to the effective parsing module set;

[0031] Among them, only the parsing modules in the effective parsing module set are allowed to be specified by the accompanying parameter information.

[0032] In one embodiment, it further includes:

[0033] After receiving the uploaded parsing module, testing the uploaded parsing module;

[0034] When the parsing module passes the test, the parsing module is included in a set of invalid parsing modules;

[0035] When the parsing module fails the test, the parsing module is discarded and prompt information is fed back.

[0036] In a second aspect, the present application provides a metadata management system for a file system, including:

[0037] A write request receiving unit, configured to receive a write request for a target file after opening the target file in the file system;

[0038] An accompanying parameter information receiving unit, configured to receive the accompanying parameter information corresponding to the write request based on a preset extended interface;

[0039] a parsing module selection unit configured to select, based on the accompanying parameter information, a parsing module specified by the accompanying parameter information from among various pre-input parsing modules for generating predefined metadata;

[0040] The metadata information generating unit is configured to write the data to be written into the target file based on the write request, and parse the data to be written by the selected parsing module to generate metadata information of the target file.

[0041] In a third aspect, the present application provides an electronic device, comprising:

[0042] memory for storing computer programs;

[0043] A processor is configured to implement the steps of the above-mentioned file system metadata management method when executing the computer program.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the metadata management method of the file system as described above are implemented.

[0045] In the solution of the present application, users can pre-upload a parsing module for generating predefined metadata. That is, if users require different metadata in different scenarios, they can pre-set a parsing module capable of generating the metadata they need and then upload it to the file system. Subsequently, when the file system opens a target file, upon receiving a write request for the target file, the parsing module specified by the accompanying parameter information can be selected based on the accompanying parameter information. Because the accompanying parameter information is required to specify the parsing module, the file system interface is extended, allowing the accompanying parameter information to be received based on a pre-set extended interface. Based on the write request, the data to be written is written to the target file, and the selected parsing module parses the data to generate metadata information for the target file, thereby generating the metadata information required by the user. As can be seen, the solution of the present application uses the specified parsing module to generate the metadata required by the user, effectively ensuring the flexibility of extended metadata. Furthermore, it can be seen that the solution can write the data to the target file and simultaneously generate the metadata information for the target file. That is, the metadata and file content are generated synchronously, ensuring the real-time metadata and consistency with the file content. This can avoid the inconsistency between the file content and metadata versions caused by asynchronous processing, which occurs in some traditional solutions.

[0046] In summary, the present application solution can effectively implement metadata management, ensure the flexibility of extending metadata, and ensure the real-time nature of metadata and its consistency with file content. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 A flowchart of a metadata management method for a file system provided in a specific embodiment of the present invention;

[0049] Figure 2 This is a functional diagram of a specific embodiment of the present invention for implementing metadata management of a file system;

[0050] Figure 3 A schematic diagram of the principle of storage control service in a specific embodiment of the present invention;

[0051] Figure 4 A schematic diagram of the structure of a metadata management system for a file system provided in a specific embodiment of the present invention;

[0052] Figure 5 A schematic structural diagram of an electronic device provided in a specific embodiment of the present invention;

[0053] Figure 6 This is a schematic structural diagram of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0055] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0056] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , Figure 1 This is a flowchart of a metadata management method for a file system provided in a specific embodiment of the present invention. The metadata management method for the file system may include the following steps:

[0057] Step S101: After opening a target file in a file system, a write request for the target file is received.

[0058] When reading or writing a target file in a file system, you first need to open the target file, that is, perform a file open operation. This can be done using the access interface provided by the file system. After opening the target file in the file system, a handle is returned. This handle can be used to find the control information for opening the file. Therefore, this handle can be used to locate the target file, allowing subsequent reading and writing operations on the target file. The target file described in this application is a file in the file system that the user currently needs to write to.

[0059] Before writing data this time, the target file can be an empty file or a file that already has some data content. It is understandable that if the target file is a file that already has data content, it means that a write operation has been performed on the target file before, so that the target file has data content stored. This time, the write operation is continued to the target file, so that data content can be added and / or modified in the target file. Of course, if it is the first time to write data to the target file, the target file will be an empty file when it is opened. Figure 2 The "open" arrow at the file shown in the example refers to opening the file.

[0060] The user can open the target file of the file system through the storage system client. After opening, the user can send a write request for the target file to the storage system client, so that the storage system client executes subsequent processes after receiving the write request.

[0061] Step S102: receiving accompanying parameter information corresponding to the write request based on a preset extended interface.

[0062] In this application, the storage system client needs to receive the accompanying parameter information corresponding to the write request. The accompanying parameter information determines which parsing module to use, which will subsequently determine the generated metadata information. To this end, the file system access interface needs to be expanded. Figure 2 ,against Figure 2 The files shown, "Open", "Close", and "Write" are all common file system access interfaces, namely, file open, close, and write. However, "Specify parsing module" needs to be implemented using an extended interface. For example, the specific implementation is an extended interface implemented through IO Ctrl, which can receive additional parameter information.

[0063] The specific content of the accompanying parameter information can be set according to actual needs, as long as the parsing module specified by the accompanying parameter information can be subsequently selected. In one specific embodiment of the present invention, the accompanying parameter information includes a parsing module identifier and an extended metadata structure. The parsing module identifier can be used to uniquely identify a parsing module from among the pre-input parsing modules, thereby identifying the parsing module specified by the accompanying parameter information. Furthermore, the accompanying parameter information in this embodiment also includes an extended metadata structure, which is used to define the representation of subsequently generated metadata information. In other words, the same metadata information can have multiple representations, and the extended metadata structure can specify these representations. For example, taking image resolution as an example, in one scenario, according to the requirements of the extended metadata structure, image resolution is specifically represented as "length: A, width: B." In another scenario, according to the requirements of the extended metadata structure, image resolution is specifically represented as "A×B." It can be seen that in this embodiment, the extended metadata structure in the accompanying parameter information can also be used to set the metadata representation format, which helps meet user needs. It is understandable that if the accompanying parameter information carries other content as described above in addition to specifying the parsing module, the accompanying parameter information can be sent to the parsing module specified by the accompanying parameter information so that the parsing module generates metadata according to the requirements of the accompanying parameter information.

[0064] Step S103: Based on the accompanying parameter information, a parsing module specified by the accompanying parameter information is selected from various pre-input parsing modules for generating predefined metadata.

[0065] The parsing module is used to generate predefined metadata, which can be pre-entered by the user. In other words, different parsing modules can be used to generate user-defined metadata, thereby meeting the needs of different users and different occasions. In other words, the parsing module can generate metadata pre-defined by the user.

[0066] Since there are multiple parsing modules, it is necessary to select the parsing module specified by the accompanying parameter information from each pre-input parsing module based on the accompanying parameter information, that is, to select the parsing module currently required by the user.

[0067] Step S104: Based on the write request, the data to be written is written into the target file, and the data to be written is parsed by the selected parsing module to generate metadata information of the target file.

[0068] Based on the write request, the data to be written may be written into the target file. In practical applications, the data to be written is usually in the form of a data stream.

[0069] In the present application, metadata is generated simultaneously with data writing. Specifically, the data to be written can be parsed by a selected parsing module to generate metadata information for the target file. This metadata information can include multiple metadata parameters, all of which can be generated by the selected parsing module. It is understood that for some metadata parameters, the parsing module requires not only the data to be written but also existing data in the target file. For example, when generating the metadata parameter "total size of the target file," it is necessary to calculate the storage space occupied by all the data in the target file.

[0070] Specifically, for the parsing module, after opening the target file, the file handle will be passed to the parsing module, so that the parsing module can create a data stream parsing context through the file handle. After that, the parsing module can call the required metadata processing functions to parse the data to be written. Of course, the specific metadata processing functions required by the parsing module can also be set in advance according to needs. The metadata processing functions required for different metadata parameters may be different. After the parsing is completed, the parsing module can return the result, that is, the metadata information of the target file is generated.

[0071] For example Figure 2 This is a functional diagram of the metadata management of the file system implemented in this application solution. Figure 2 In the example, based on the accompanying parameter information, the user specifically specifies parsing module 1, then the storage system client specifically selects parsing module 1 through the storage control service of the storage node, and passes the write request to the storage control service. Then, through the metadata service of the storage node, the data to be written is parsed through parsing module 1, and the metadata information of the target file is finally generated. It can be seen that after the storage control service receives the write request, the data stream is written to the storage system backend storage pool, and the metadata processing function is called through the parsing module to return the actively parsed metadata information, that is, the metadata information of the target file that meets the user's customized requirements is returned. In addition, the complexity of the client can be reduced. The application does not need to integrate multiple file parsing libraries (such as PDF parsing, image EXIF ​​extraction), but can directly select the required parsing module through instructions to trigger the automatic processing of the storage system.

[0072] In actual applications, metadata information of the target file may be stored after being generated, and active metadata parsing subscription events may be pushed for upper-layer applications to subscribe to and trigger subsequent operations of the upper-layer applications based on the metadata information.

[0073] See Figure 3, which is a schematic diagram of the storage control service principle. After the write process is triggered, data stream parsing will be performed. While the data is being written to the disk, metadata is generated through the selected parsing module.

[0074] In a specific embodiment of the present invention, after generating metadata information of the target file, the method further includes:

[0075] Storing the metadata information of the currently generated target file in a storage manner that does not overwrite the metadata information of the old version of the target file;

[0076] Generate a version number for the metadata information of the currently generated target file, and fill the version number into the metadata version record table corresponding to the target file.

[0077] This implementation further takes into account that, since the present application solution allows metadata information that meets user-defined requirements to be generated through the corresponding parsing module in accordance with user-defined requirements, in order to avoid the loss of metadata required by the user, in this implementation, the metadata information of the currently generated target file is stored in a storage method that does not overwrite the old version of the metadata information of the target file, and a version number is generated for the metadata information of the currently generated target file, which is maintained in the metadata version record table of the target file.

[0078] In a specific embodiment of the present invention, it may further include:

[0079] receiving a metadata version selection instruction for a target file;

[0080] Retain the version number specified by the metadata version selection instruction in the metadata version record table of the target file and delete the remaining version numbers;

[0081] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0082] This implementation takes into account that since different metadata versions of the target file are stored, it supports deleting old and useless metadata versions to avoid wasting storage resources. Specifically, for example, after detecting a user operation on the operation interface and determining that the user has selected a certain version, a metadata version selection instruction can be generated and sent to the storage system client. The storage system client can then, based on the received metadata version selection instruction for the target file, retain the version number specified by the metadata version selection instruction in the metadata version record table of the target file, and delete the remaining version numbers. It is understood that for any deleted version number, the stored metadata information corresponding to that version number needs to be deleted.

[0083] It can be seen that in this implementation, users are allowed to selectively submit valid versions, that is, the version number specified by the metadata version selection instruction is regarded as the valid version number selected by the user, and the metadata versions corresponding to the remaining version numbers can be deleted without being retained to avoid wasting storage resources.

[0084] Furthermore, in a specific embodiment of the present invention, it may also include:

[0085] When it is detected that the metadata version record table of the target file meets the set trigger conditions, the automatic deletion verification information is output to the designated terminal;

[0086] The metadata version meets the trigger condition when the total number of version numbers in the metadata version record table reaches a set first number threshold, and / or when the number of version numbers that have existed in the metadata version record table for a first time period reaches a set second threshold;

[0087] When a first feedback instruction indicating prohibition of automatic deletion is received, the automatic deletion process is terminated;

[0088] When a second feedback instruction indicating that automatic deletion is allowed is received, and / or when the first feedback instruction is not received within the waiting time, all version numbers in the metadata version record table of the target file that have existed for a period of time equal to the first time period are deleted;

[0089] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0090] This implementation takes into account that each time a write operation is performed on a target file, the storage system generates and stores metadata information for the target file's current version in accordance with the principles of this application. Therefore, over time, the storage space occupied by metadata may be considerable. This is especially true in scenarios where there are many files and different files have multiple versions of metadata, making the storage space occupied by metadata even more significant. Although the user is allowed to select the desired version as described above, the user may not necessarily select a version for each file in a timely manner.

[0091] This embodiment provides an automatic cleanup mechanism. Specifically, when the total number of version numbers in the metadata version record table of the target file reaches a set first threshold, it indicates that many versions of metadata of the target file have been recorded. Therefore, it can be determined that the metadata version record table of the target file meets the set trigger condition. In addition, if the number of version numbers that have existed for a first duration in the metadata version record table reaches a set second threshold, it indicates that the metadata of the target file has many versions and is very old data. Therefore, it can also be determined that the metadata version record table of the target file meets the set trigger condition. In addition, to determine the existence duration of a version number, it can usually be determined based on the timestamp of the version number. In some implementation methods, in order to facilitate verification, in addition to recording each version number, the metadata version record table of the target file can also record information such as the timestamp when each version was generated.

[0092] When the triggering conditions are met, the automatic deletion verification information can be output to the designated terminal. The designated terminal can be, for example, the mobile phone, computer or other terminal device of the designated staff, or it can be a special platform where the staff can view the content of the platform through web login or other methods.

[0093] When a first feedback instruction indicating prohibition of automatic deletion is received, it indicates that the staff believes that the metadata of different versions of the target file are mostly useful and cannot be deleted temporarily, so the automatic deletion process can be directly terminated. When a second feedback instruction indicating permission for automatic deletion is received, and / or if the first feedback instruction is not received within the waiting period, it indicates that deletion of some old versions of metadata is permitted. In this embodiment, all version numbers in the metadata version record table of the target file that have existed for a period of time exceeding the first period will be deleted. Accordingly, for any deleted version number, the stored metadata information corresponding to the version number needs to be deleted.

[0094] It can be seen that this implementation method can realize automatic cleaning of metadata versions, which reduces the workload of staff compared to periodic and proactive cleaning by staff. In addition, the automatic cleaning mechanism deletes old versions of metadata, so it can reduce the probability of important metadata versions being deleted to a certain extent.

[0095] In a specific embodiment of the present invention, it may further include:

[0096] A correspondence between a designated field in the metadata information of the target file and a file path of the target file is established so that the content in the designated field serves as index data of the target file.

[0097] In this implementation, a correspondence can be established between a specified field in the metadata information of the target file and the file path of the target file, thereby achieving indexing. In particular, for the retrieval of unstructured data, this can effectively improve the flexibility and convenience of retrieval, thereby achieving efficient retrieval and data governance. Taking the specified field of resolution as an example, after establishing a correspondence between the specified field and the file path of the target file, for example, if the specific content of the specified field in the metadata information of the target file is A×B, then the target file can be determined by searching A×B, that is, the file path with the target file can be located.

[0098] In a specific embodiment of the present invention, it may further include:

[0099] Receive the uploaded parsing module and list the parsing module in the set of ineffective parsing modules;

[0100] Receive the parsing module validation instruction;

[0101] Move the parsing module specified by the parsing module validation instruction from the ineffective parsing module set to the effective parsing module set;

[0102] Among them, only the parsing modules in the effective parsing module set are allowed to be specified by the accompanying parameter information.

[0103] In this implementation, the storage system client can receive the uploaded parsing module, for example Figure 2 In the example, the storage system client receives Parsing Module 3 uploaded by the user. Furthermore, in this implementation, two sets are maintained: an ineffective parsing module set and an effective parsing module set. After receiving the uploaded parsing module, the parsing module is added to the ineffective parsing module set. Parsing modules in the ineffective parsing module set are not allowed to be used. In other words, in this implementation, when selecting a parsing module specified by the accompanying parameter information from among the pre-input parsing modules, the selection can only be made from the effective parsing module set.

[0104] When a parsing module validation instruction is received, the parsing module specified by the parsing module validation instruction may be moved from the ineffective parsing module set to the effective parsing module set according to the requirements of the parsing module validation instruction.

[0105] Furthermore, it is understandable that when a parsing module invalidation instruction is received, the parsing module specified by the parsing module invalidation instruction may be moved from the valid parsing module set to the invalid parsing module set according to the requirements of the parsing module invalidation instruction.

[0106] Of course, some implementations also support deletion of a parsing module. That is, when a parsing module deletion instruction is received, the parsing module specified by the parsing module deletion instruction can be deleted from the ineffective parsing module set or the effective parsing module set.

[0107] In actual application, for the convenience of management, you can create a list of valid parsing modules and a list of invalid parsing modules, for example Figure 3 In the example, the effective parsing module list includes parsing module 1, parsing module 2 and parsing module 3.

[0108] This implementation method takes into account that there may be a large number of parsing modules, some of which may be of older versions. To facilitate management, a set of ineffective parsing modules and a set of effective parsing modules are maintained. By moving parsing modules from the ineffective parsing module set to the effective parsing module set, or vice versa, management convenience is improved. For example, if a parsing module is older and no longer suitable for current needs, the user can move it from the effective parsing module set to the ineffective parsing module set. After the user subsequently adjusts it, the module can be moved back to the effective parsing module set after the ineffective parsing module set.

[0109] In a specific embodiment of the present invention, it may further include:

[0110] After receiving the uploaded parsing module, testing the uploaded parsing module;

[0111] When the parsing module passes the detection, the operation of adding the parsing module to the set of invalid parsing modules is executed;

[0112] When the parsing module fails the test, the parsing module is discarded and a prompt message is fed back.

[0113] This implementation method takes into account the large number of parsing modules. Therefore, after receiving the uploaded parsing modules, the uploaded parsing modules can be tested, so as to discover some parsing modules with problems as early as possible. When the parsing module passes the test, the operation of listing it in the set of invalid parsing modules can be performed normally. Correspondingly, when the parsing module fails the test, the parsing module can be discarded and a prompt message can be fed back.

[0114] The uploaded parsing module is tested. The specific test items can be set and adjusted according to actual needs. For example, in a specific scenario, the interface parameters of the metadata processing function used by the parsing module can be tested to see whether they meet the interface parameter requirements. The data structure when the parsing module generates metadata can be tested to see whether it meets the data structure requirements. The format of the generated metadata can be tested to see whether it is consistent with the system requirements, etc.

[0115] In the solution of the present application, users can pre-upload a parsing module for generating predefined metadata. That is, if users require different metadata in different scenarios, they can pre-set a parsing module capable of generating the metadata they need and then upload it to the file system. Subsequently, when the file system opens a target file, upon receiving a write request for the target file, the parsing module specified by the accompanying parameter information can be selected based on the accompanying parameter information. Because the accompanying parameter information is required to specify the parsing module, the file system interface is extended, allowing the accompanying parameter information to be received based on a pre-set extended interface. Based on the write request, the data to be written is written to the target file, and the selected parsing module parses the data to generate metadata information for the target file, thereby generating the metadata information required by the user. As can be seen, the solution of the present application uses the specified parsing module to generate the metadata required by the user, effectively ensuring the flexibility of extended metadata. Furthermore, it can be seen that the solution can write the data to the target file and simultaneously generate the metadata information for the target file. That is, the metadata and file content are generated synchronously, ensuring the real-time metadata and consistency with the file content. This can avoid the inconsistency between the file content and metadata versions caused by asynchronous processing, which occurs in some traditional solutions.

[0116] In summary, the present application solution can effectively implement metadata management, ensure the flexibility of extending metadata, and ensure the real-time nature of metadata and its consistency with file content.

[0117] Corresponding to the above method embodiment, an embodiment of the present invention further provides a metadata management system for a file system, which can be referred to in correspondence with the above.

[0118] See also Figure 4 FIG. 1 is a schematic diagram of a metadata management system for a file system according to the present invention, comprising:

[0119] A write request receiving unit 401 is configured to receive a write request for a target file after opening the target file in the file system;

[0120] The accompanying parameter information receiving unit 402 is configured to receive the accompanying parameter information corresponding to the write request based on a preset extended interface;

[0121] The parsing module selection unit 403 is configured to select, based on the accompanying parameter information, a parsing module specified by the accompanying parameter information from among the pre-input parsing modules for generating predefined metadata;

[0122] The metadata information generating unit 404 is configured to write the data to be written into the target file based on the write request, and parse the data to be written by the selected parsing module to generate metadata information of the target file.

[0123] In a specific embodiment of the present invention, a version recording unit is further included, which is used to:

[0124] Storing the currently generated metadata information of the target file in a storage manner that does not overwrite the metadata information of the old version of the target file;

[0125] A version number is generated for the metadata information of the target file currently being generated, and the version number is filled into a metadata version record table corresponding to the target file.

[0126] In a specific embodiment of the present invention, a metadata active deletion unit is further included, which is used to:

[0127] receiving a metadata version selection instruction for the target file;

[0128] retaining, in the metadata version record table of the target file, the version number specified by the metadata version selection instruction and deleting the remaining version numbers;

[0129] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0130] In a specific embodiment of the present invention, the system further includes a metadata automatic deletion unit configured to:

[0131] When it is detected that the metadata version record table of the target file meets the set trigger condition, outputting automatic deletion verification information to a designated terminal;

[0132] When the total number of version numbers in the metadata version record table reaches a set first number threshold, and / or when the number of version numbers in the metadata version record table that have existed for a first time period reaches a set second threshold, the metadata version meets the trigger condition;

[0133] When a first feedback instruction indicating prohibition of automatic deletion is received, the automatic deletion process is terminated;

[0134] When a second feedback instruction indicating that automatic deletion is allowed is received, and / or when the first feedback instruction is not received within the waiting time, all version numbers of the target file that have existed in the metadata version record table for a period of time exceeding the first time period are deleted;

[0135] For any deleted version number, the stored metadata information corresponding to the version number is deleted.

[0136] In a specific embodiment of the present invention, an index building unit is further included, which is used to:

[0137] A correspondence between a designated field in the metadata information of the target file and a file path of the target file is established, so that the content in the designated field serves as index data of the target file.

[0138] In a specific embodiment of the present invention, a parsing module management unit is further included, which is used to:

[0139] Receiving the uploaded parsing module and listing the parsing module in a set of ineffective parsing modules;

[0140] Receive the parsing module validation instruction;

[0141] Moving the parsing module specified by the parsing module validation instruction from the ineffective parsing module set to the effective parsing module set;

[0142] Among them, only the parsing modules in the effective parsing module set are allowed to be specified by the accompanying parameter information.

[0143] In a specific embodiment of the present invention, a parsing module detection unit is further included, which is used to:

[0144] After receiving the uploaded parsing module, testing the uploaded parsing module;

[0145] When the parsing module passes the test, the parsing module is included in a set of invalid parsing modules;

[0146] When the parsing module fails the test, the parsing module is discarded and prompt information is fed back.

[0147] Corresponding to the above method and system embodiments, embodiments of the present invention further provide an electronic device, a computer-readable storage medium, and a computer program product, which can be referred to in correspondence with the above.

[0148] See also Figure 5 As shown, the device may include:

[0149] Memory 501, used for storing computer programs;

[0150] The processor 502 is configured to execute a computer program to implement the steps of the metadata management method for a file system in any of the above embodiments.

[0151] The computer program product includes a computer program / instruction, which implements the steps of the metadata management method for a file system in any of the above embodiments when executed by a processor.

[0152] See Figure 6 The computer-readable storage medium 60 stores a computer program 61. When executed by a processor, the computer program 61 implements the steps of the file system metadata management method described in any of the above embodiments. The computer-readable storage medium 60 herein includes random access memory (RAM), internal memory, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, or any other form of storage medium known in the art.

[0153] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0154] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only intended to help understand the technical solutions and core concepts of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications also fall within the scope of protection of the present invention.

Claims

1. A metadata management method for a file system, characterized in that: include: After opening a target file in the file system, receiving a write request for the target file; Receiving accompanying parameter information corresponding to the write request based on a preset extended interface; Based on the accompanying parameter information, selecting a parsing module specified by the accompanying parameter information from various pre-input parsing modules for generating predefined metadata; Based on the write request, the data to be written is written into the target file, and the data to be written is parsed by the selected parsing module to generate metadata information of the target file.

2. The metadata management method of a file system according to claim 1, characterized in that: After generating the metadata information of the target file, the method further includes: Storing the currently generated metadata information of the target file in a storage manner that does not overwrite the metadata information of the old version of the target file; A version number is generated for the metadata information of the target file currently being generated, and the version number is filled into a metadata version record table corresponding to the target file.

3. The metadata management method of a file system according to claim 2, characterized in that: Also includes: receiving a metadata version selection instruction for the target file; retaining, in the metadata version record table of the target file, the version number specified by the metadata version selection instruction and deleting the remaining version numbers; For any deleted version number, the stored metadata information corresponding to the version number is deleted.

4. The metadata management method of a file system according to claim 3, wherein: Also includes: When it is detected that the metadata version record table of the target file meets the set trigger condition, outputting automatic deletion verification information to a designated terminal; When the total number of version numbers in the metadata version record table reaches a set first number threshold, and / or when the number of version numbers in the metadata version record table that have existed for a first time period reaches a set second threshold, the metadata version meets the trigger condition; When a first feedback instruction indicating prohibition of automatic deletion is received, the automatic deletion process is terminated; When a second feedback instruction indicating that automatic deletion is allowed is received, and / or when the first feedback instruction is not received within the waiting time, all version numbers of the target file that have existed in the metadata version record table for a period of time exceeding the first time period are deleted; For any deleted version number, the stored metadata information corresponding to the version number is deleted.

5. The metadata management method of a file system according to claim 1, wherein: Also includes: A correspondence between a designated field in the metadata information of the target file and a file path of the target file is established, so that the content in the designated field serves as index data of the target file.

6. The metadata management method for a file system according to any one of claims 1 to 5, characterized in that: Also includes: receiving the uploaded parsing module and listing the parsing module in a set of ineffective parsing modules; Receive the parsing module validation instruction; Moving the parsing module specified by the parsing module validation instruction from the ineffective parsing module set to the effective parsing module set; Among them, only the parsing modules in the effective parsing module set are allowed to be specified by the accompanying parameter information.

7. The metadata management method of a file system according to claim 6, characterized in that: Also includes: After receiving the uploaded parsing module, the uploaded parsing module is tested; When the parsing module passes the test, the parsing module is included in a set of invalid parsing modules; When the parsing module fails the test, the parsing module is discarded and prompt information is fed back.

8. A metadata management system for a file system, characterized in that: include: A write request receiving unit, configured to receive a write request for a target file after opening the target file in the file system; An accompanying parameter information receiving unit, configured to receive the accompanying parameter information corresponding to the write request based on a preset extended interface; a parsing module selection unit configured to select, based on the accompanying parameter information, a parsing module specified by the accompanying parameter information from among various pre-input parsing modules for generating predefined metadata; The metadata information generating unit is configured to write the data to be written into the target file based on the write request, and parse the data to be written by the selected parsing module to generate metadata information of the target file.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the file system metadata management method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the metadata management method for a file system according to any one of claims 1 to 7 are implemented.

Images