File indexing method and apparatus, storage medium, and electronic device

Abstract

The present disclosure relates to the technical field of computer, in particular to a file indexing method, a file indexing device, a storage medium and an equipment, the method comprising: in response to a file indexing request, obtaining an indexing level; obtaining a plurality of first indexing directories corresponding to the indexing level; calculating a hash value for each first indexing subdirectory, and determining a plurality of files in the first indexing subdirectory according to the hash value; calling metadata information corresponding to the plurality of files, and searching for the files corresponding to the plurality of metadata information according to the plurality of metadata information. Through the technical scheme of the embodiment of the present disclosure, the problem of poor indexing efficiency in related technologies can be solved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to a file indexing method, a file indexing device, a computer-readable storage medium, and an electronic device. Background Technology

[0002] With the rapid development of software and hardware, data is expanding rapidly, and various file management systems are constantly being developed. File management systems can handle massive amounts of data, enabling searching and management. For example, HPC (High Performance Computing) systems possess high computing power, high storage capacity, high bandwidth, high parallelism, and high reliability, and can be used in fields requiring high-intensity computing, such as science, engineering, and medicine.

[0003] However, solutions in related technologies typically employ spatial tree architectures or use external databases to index data. When the number of files in the file system is very large, using these traditional indexing techniques can lead to a decrease in the performance of data querying and indexing.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a file indexing method, file indexing device, computer-readable storage medium, and electronic device that can solve the problem of poor indexing efficiency in related technologies.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to a first aspect of this disclosure, a file indexing method is provided, comprising: in response to a file indexing request, obtaining an index level; wherein the index level is any index level other than the index level corresponding to the root directory; obtaining a plurality of first index directories corresponding to the index level; wherein the first index directories include at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; calculating a hash value for each first index subdirectory, and determining a plurality of files in the first index subdirectory based on the hash value; and calling metadata information corresponding to the plurality of files, and searching for the files corresponding to the plurality of metadata information based on the plurality of metadata information.

[0008] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, obtaining multiple first index directories corresponding to the index level includes: determining whether the file to be indexed is in the multiple first index directories corresponding to the index level; if the file to be indexed is in the multiple first index directories corresponding to the index level, obtaining the multiple first index directories corresponding to the index level; if the file to be indexed is not in the multiple first index directories corresponding to the index level, executing the root directory process; wherein, the root directory process is the process that indexes the file starting from the root directory.

[0009] In one exemplary embodiment of this disclosure, based on the foregoing scheme, calculating a hash value for each first index subdirectory includes: detecting whether each first index subdirectory contains data; when each first index subdirectory contains data, calculating a hash value for each first index subdirectory; when each first index subdirectory does not contain data, executing a root directory process; wherein, the root directory process is a process that indexes files starting from the root directory.

[0010] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, the root directory process includes: obtaining the root directory index level; wherein, the root directory index level is the index level corresponding to the root directory; obtaining the root directory corresponding to the root directory index level; wherein, the root directory includes at least one second index subdirectory at an index sublevel, and each index sublevel includes at least one second index subdirectory; calculating a hash value for each second index subdirectory, and determining multiple files in the second index subdirectory based on the hash value; calling the metadata information corresponding to the multiple files, and searching for the files corresponding to the multiple metadata information based on the multiple metadata information.

[0011] In one exemplary embodiment of this disclosure, based on the foregoing scheme, file indexing requests are sent through different clients, each with a different index directory.

[0012] In one exemplary embodiment of this disclosure, based on the aforementioned scheme, a file change log is obtained, and change-related information is extracted from the file change log; wherein, the file change log is used to record file change events, and the change-related information includes the file identifier corresponding to the file; the file identifier is parsed into an absolute path, and it is detected whether the cache includes the file identifier; if the cache does not include the file identifier, the file identifier is parsed to obtain the identifier path, and the identifier path is added to the suspicious file; if the cache includes the file identifier, the absolute path is added to the suspicious file, so as to improve the indexing frequency through the suspicious file.

[0013] In one exemplary embodiment of this disclosure, based on the foregoing scheme, the file change log includes change events for multiple files, and the method further includes: after the change events for multiple files have been processed, clearing the change events for multiple files from the file change log.

[0014] According to a second aspect of this disclosure, a file indexing apparatus is provided, comprising: an index level acquisition module, configured to acquire an index level in response to a file indexing request; wherein the index level is any index level other than the index level corresponding to the root directory; an index directory acquisition module, configured to acquire a plurality of first index directories corresponding to the index level; wherein the first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; a hash value calculation module, configured to calculate a hash value for each first index subdirectory, and determine a plurality of files in the first index subdirectory based on the hash value; and a metadata retrieval module, configured to retrieve metadata information corresponding to the plurality of files, and locate the files corresponding to the plurality of metadata information based on the plurality of metadata information.

[0015] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the file indexing method of the first aspect of the above embodiments.

[0016] According to a fourth aspect of this disclosure, an electronic device is provided, comprising:

[0017] One or more processors; and

[0018] A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the file indexing method of the first aspect in the above embodiments.

[0019] The technical solutions provided in this disclosure can include the following beneficial effects:

[0020] In one embodiment of the file indexing method provided by this disclosure, in response to a file indexing request, an index level can be obtained, multiple first index directories corresponding to the index level can be obtained, a hash value can be calculated for each first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, metadata information corresponding to the multiple files can be retrieved, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. Through the embodiments of this disclosure, when performing an indexing task, any index level can be obtained, and the metadata information of files can be searched in the directory corresponding to the index level, and the files can be found based on the metadata information, which can improve the efficiency of indexing and thus improve the efficiency of file management.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0023] Figure 1 The schematic diagram illustrates an exemplary system architecture of the file indexing method in an exemplary embodiment of this disclosure;

[0024] Figure 2 A flowchart illustrating a file indexing method in an exemplary embodiment of this disclosure is shown schematically.

[0025] Figure 3 This schematically illustrates a flowchart of the root directory process when the file to be indexed is not in one of the multiple first index directories corresponding to the index level in an exemplary embodiment of this disclosure.

[0026] Figure 4 This schematically illustrates a flowchart of the root directory process when no data is contained in each of the first index subdirectories in an exemplary embodiment of the present disclosure;

[0027] Figure 5 This schematically illustrates a flowchart of a process for finding a file corresponding to multiple metadata information based on multiple metadata information in an exemplary embodiment of this disclosure;

[0028] Figure 6 This schematic diagram illustrates the different index directories corresponding to each client in an exemplary embodiment of this disclosure;

[0029] Figure 7 A flowchart illustrating another file indexing method in an exemplary embodiment of this disclosure is shown schematically.

[0030] Figure 8 This schematically illustrates a flowchart of adding an absolute path to a suspicious file when a file identifier is included in the cache, as shown in an exemplary embodiment of this disclosure.

[0031] Figure 9 This schematically illustrates a flowchart of generating a suspicious file in an exemplary embodiment of the present disclosure;

[0032] Figure 10 This schematic diagram illustrates a reindexer setup in an exemplary embodiment of the present disclosure.

[0033] Figure 11 This schematic diagram illustrates a distributed storage system according to an exemplary embodiment of the present disclosure;

[0034] Figure 12This schematic diagram illustrates the composition of a document indexing apparatus in an exemplary embodiment of the present disclosure;

[0035] Figure 13 The schematic diagram illustrates a structural schematic of a computer system suitable for implementing an electronic device according to exemplary embodiments of the present disclosure. Detailed Implementation

[0036] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., may be employed. In other instances, well-known structures, methods, apparatuses, implementations, materials, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0037] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more software-hardened modules, or in different network and / or processor devices and / or microcontroller devices.

[0038] Figure 1 A schematic diagram of an exemplary system architecture to which the file indexing method of embodiments of this disclosure can be applied is shown.

[0039] like Figure 1 As shown, system architecture 1000 may include one or more of terminal devices 1001, 1002, and 1003, network 1004, and server 1005. Network 1004 is used as a medium to provide a communication link between terminal devices 1001, 1002, and 1003 and server 1005. Network 1004 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0040] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. For example, server 1005 could be a server cluster composed of multiple servers.

[0041] Users can use terminal devices 1001, 1002, and 1003 to interact with server 1005 via network 1004 to receive or send messages, etc. Terminal devices 1001, 1002, and 1003 can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. Additionally, server 1005 can be a server providing various services.

[0042] In one embodiment, the execution entity of the file indexing method disclosed herein may be a server 1005. The server 1005 may obtain file indexing requests sent by terminal devices 1001, 1002, and 1003, obtain the index level, obtain multiple first index directories corresponding to the index level, calculate hash values ​​for each first index subdirectory, determine multiple files in the first index subdirectory based on the hash values, call the metadata information corresponding to the multiple files, and search for the files corresponding to the multiple metadata information based on the multiple metadata information.

[0043] In addition, the file indexing method disclosed herein can be executed through terminal devices 1001, 1002, 1003, etc., to obtain a file index request, obtain an index level, obtain multiple first index directories corresponding to the index level, calculate a hash value for each first index subdirectory, determine multiple files in the first index subdirectory based on the hash value, call the metadata information corresponding to the multiple files, and find the files corresponding to the multiple metadata information based on the multiple metadata information.

[0044] Furthermore, the implementation process of this document indexing method can also be jointly implemented by terminal devices 1001, 1002, and 1003 and server 1005. For example, terminal devices 1001, 1002, and...

[0045] Server 1003 can obtain a file index request, obtain the index level, and then send the determined index level to server 1005 so that server 1005 can obtain multiple first index directories corresponding to the index level, calculate the hash value for each first index subdirectory, determine multiple files in the first index subdirectory based on the hash value, call the metadata information corresponding to the multiple files, and find the files corresponding to the multiple metadata information based on the multiple metadata information.

[0046] With the rapid development of software and hardware, data is expanding rapidly, and various file management systems are constantly being developed. File management systems can handle massive amounts of data, enabling searching and management. For example, HPC (High Performance Computing) systems possess high computing power, high storage capacity, high bandwidth, high parallelism, and high reliability, and can be used in fields requiring high-intensity computing, such as science, engineering, and medicine.

[0047] However, solutions in related technologies typically employ spatial tree architectures or use external databases to index data. When the number of files in the file system is very large, using these traditional indexing techniques can lead to a decrease in the performance of data querying and indexing.

[0048] In the file indexing method provided in this exemplary embodiment, in response to a file indexing request, an index level can be obtained, multiple first index directories corresponding to the index level can be obtained, a hash value can be calculated for each first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, metadata information corresponding to the multiple files can be called, and files corresponding to the multiple metadata information can be found based on the multiple metadata information. For example... Figure 2 As shown, the file indexing method may include the following steps:

[0049] Step S210: In response to the file indexing request, obtain the index level; wherein, the file indexing request is used to indicate the file to be indexed, and the index level is any index level other than the index level corresponding to the root directory;

[0050] Step S220: Obtain multiple first index directories corresponding to the index level; wherein, the first index directory includes at least one first index subdirectory of the index sublevel, and each index sublevel includes at least one first index subdirectory;

[0051] Step S230: Calculate the hash value for each first index subdirectory, and determine multiple files in the first index subdirectory based on the hash value;

[0052] Step S240: Call the metadata information corresponding to multiple files, and find the file corresponding to the multiple metadata information based on the multiple metadata information.

[0053] In one embodiment of the file indexing method provided by this disclosure, in response to a file indexing request, an index level can be obtained, multiple first index directories corresponding to the index level can be obtained, a hash value can be calculated for each first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, metadata information corresponding to the multiple files can be retrieved, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. Through the embodiments of this disclosure, when performing an indexing task, any index level can be obtained, and the metadata information of files can be searched in the directory corresponding to the index level, and the files can be found based on the metadata information, which can improve the efficiency of indexing.

[0054] The steps S210 to S240 of the file indexing method in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.

[0055] Step S210: In response to the file indexing request, obtain the index level;

[0056] In one example embodiment of this disclosure, an index level can be obtained in response to a file indexing request. The file indexing request indicates the file to be indexed, and the index level can be any index level other than the index level corresponding to the root directory. Specifically, the root directory refers to the top-level directory, which is also the starting directory for all other directories and files. The obtained index level is not necessarily the index level corresponding to the root directory. When an indexing task is required, a file indexing request can be triggered to obtain the index level to be indexed. The file indexing request can include the index level to be indexed, and the index level can be used to indicate the level of the relationship between the index and the data.

[0057] Specifically, a file management system can include multiple index levels.

[0058] It should be noted that this disclosure does not impose any special restrictions on the method of obtaining the index level.

[0059] For example, you can input the index level through the Indexer.

[0060] In one example embodiment of this disclosure, the scheme of this disclosure can be implemented using Lustre (parallel distributed file system).

[0061] Step S220: Obtain multiple first index directories corresponding to the index level;

[0062] In one example embodiment of this disclosure, after obtaining the index level through the above steps, multiple first index directories corresponding to the index level can be obtained. Each first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory. Specifically, after determining the index level, multiple first index directories corresponding to the index level can be searched; that is, each index level can correspond to multiple first index directories, and each first index directory also includes multiple first index subdirectories.

[0063] Specifically, the first index directory may also include multiple first index subdirectories at the index sub-level, and all subdirectories at each index sub-level are first index subdirectories.

[0064] For example, the root directory includes a first directory, a second directory, and a third directory. The first directory includes a first subdirectory, a second subdirectory, and a third subdirectory. The second directory includes a fourth subdirectory, a fifth subdirectory, and a sixth subdirectory. The third directory includes a seventh subdirectory, an eighth subdirectory, and a ninth subdirectory. The root directory has the first index level, the first, second, and third subdirectories have the second index level, and the first to ninth subdirectories have the third index level. The first index level is greater than the second index level, which is greater than the third index level. The obtained index level is the second index level. That is, the multiple first index directories corresponding to the second index level include the first, second, and third subdirectories, and the first index directory itself includes multiple first index subdirectories (first to ninth subdirectories). Furthermore, if the seventh subdirectory also includes the tenth and eleventh subdirectories, then the multiple first index subdirectories included in the first index directory are the first to eleventh subdirectories.

[0065] It should be noted that this disclosure does not impose any special restrictions on the specific methods for obtaining multiple first index directories corresponding to the index level.

[0066] Specifically, the first indexed subdirectory can be determined through recursive checking. Recursive checking refers to searching for or matching a specific element or pattern in a data structure. It is typically implemented by recursively calling functions or methods. In each recursive call, an element or child node in the current data structure is checked, and the result determines whether to continue recursively. For example, the first indexed subdirectory can be determined using `readdir()` (a recursive algorithm).

[0067] In one example embodiment of this disclosure, after obtaining the index level through the above steps, it can be determined whether the index level of the file to be indexed is the same as the obtained index level. If they are the same, multiple first index directories corresponding to the index level are obtained, a hash value is calculated for each first index subdirectory, multiple files in the first index subdirectory are determined based on the hash value, metadata information corresponding to the multiple files is called, and files corresponding to the multiple metadata information are found based on the multiple metadata information. If they are not the same, the root directory process is executed. Here, the first index directory includes at least one first index subdirectory at each index sublevel, each index sublevel includes at least one first index subdirectory, and the root directory process is the process that indexes files starting from the root directory.

[0068] Step S230: Calculate the hash value for each first index subdirectory, and determine multiple files in the first index subdirectory based on the hash value;

[0069] In one example embodiment of this disclosure, after obtaining multiple first index subdirectories through the above steps, a hash value can be calculated for each first index subdirectory, and multiple files in the first index subdirectory can be determined based on the hash value. Specifically, when calculating the hash value for a first index subdirectory, the hash value can be calculated based on the data in the first index subdirectory. After obtaining the hash value of the first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, thus determining the files that can be accessed by the first index subdirectory.

[0070] For example, the hash value of each first index subdirectory can be calculated using MD5 (Message-Digest Algorithm 5).

[0071] It should be noted that this disclosure does not impose any special restrictions on the specific method for determining multiple files in the first index subdirectory based on hash values.

[0072] Step S240: Call the metadata information corresponding to multiple files, and find the file corresponding to the multiple metadata information based on the multiple metadata information.

[0073] In one example embodiment of this disclosure, after determining multiple files in the first index subdirectory through the above steps, the metadata information corresponding to the multiple files can be retrieved, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. Specifically, the metadata information corresponding to the files refers to data describing the data, which can be used to describe and explain the data itself, including information such as the data's attributes, characteristics, structure, content, source, and format. After obtaining the metadata information corresponding to multiple files, the files corresponding to the multiple metadata information can be found through the metadata information. For example, the metadata information includes the storage path of the file on the storage server, and the files corresponding to the multiple metadata information can be found on the storage server through this storage path.

[0074] For example, the new_lstat(metadata call) method can be used to retrieve metadata information for multiple files.

[0075] For example, the metadata information corresponding to a file can be inode (index node) information.

[0076] It should be noted that this disclosure does not impose any special restrictions on the specific methods for calling metadata information corresponding to multiple files or for finding files corresponding to multiple metadata information based on multiple metadata information.

[0077] In one example embodiment of this disclosure, after determining multiple files in the first index subdirectory based on hash values, the metadata information corresponding to the multiple files can be invoked. If the invocation is successful, the file corresponding to the multiple metadata information is searched based on the multiple metadata information; if the invocation fails, the root directory process is executed. The root directory process is the process that indexes files starting from the root directory.

[0078] In one example embodiment of this disclosure, it can be determined whether the file to be indexed is in one of the multiple first index directories corresponding to the index level. If the file to be indexed is in one of the multiple first index directories corresponding to the index level, the multiple first index directories corresponding to the index level are obtained. If the file to be indexed is not in one of the multiple first index directories corresponding to the index level, the root directory process is executed. (Refer to...) Figure 3 As shown, if the file to be indexed is not in any of the multiple first index directories corresponding to the index level, the root directory process can be executed, which may include the following steps S310 to S330:

[0079] Step S310: Determine whether the file to be indexed is in one of the multiple first index directories corresponding to the index level;

[0080] Step S320: If the file to be indexed is in multiple first index directories corresponding to the index level, obtain the multiple first index directories corresponding to the index level;

[0081] Step S330: If the file to be indexed is not in any of the multiple first index directories corresponding to the index level, execute the root directory process;

[0082] In one example embodiment of this disclosure, after obtaining the index level through the above steps, it can be first determined whether the file to be indexed is in one of the multiple first index directories corresponding to the index level. If the file to be indexed is in one of the multiple first index directories corresponding to the index level, the multiple first index directories corresponding to the index level can be obtained, and the above file indexing method can continue to be executed. If the file to be indexed is not in one of the multiple first index directories corresponding to the index level, it means that the file to be indexed is not contained in one of the multiple first index directories corresponding to the index level. At this time, it is necessary to go back to the root directory for indexing, that is, to execute the root directory process. Specifically, the root directory process is the process of indexing files starting from the root directory.

[0083] It should be noted that this disclosure does not impose any special restrictions on the specific process of the root directory process.

[0084] Through the above steps S310 to S330, it can be determined whether the file to be indexed is in one of the multiple first index directories corresponding to the index level. If the file to be indexed is in one of the multiple first index directories corresponding to the index level, the multiple first index directories corresponding to the index level are obtained. If the file to be indexed is not in one of the multiple first index directories corresponding to the index level, the root directory process is executed. Through the embodiments of this disclosure, when starting indexing, indexing can begin from any of the multiple first index directories corresponding to any index level other than the root directory. If the file to be indexed is in one of the first index directories, the indexing efficiency can be improved. If the file to be indexed is not in one of the multiple first index directories, the indexing process returns to the root directory.

[0085] In one example embodiment of this disclosure, after obtaining the index level through the above steps, a data table can be created in the first index directory, and the data table can be used to record the indexing process.

[0086] In one example embodiment of this disclosure, it is possible to detect whether each first index subdirectory contains data. If data is present in each first index subdirectory, a hash value is calculated for each first index subdirectory. If no data is present in each first index subdirectory, the root directory process is executed. (Refer to...) Figure 4 As shown, when no data is contained in any of the first index subdirectories, executing the root directory process may include the following steps S410 to S430:

[0087] Step S410: Check whether each first index subdirectory contains data;

[0088] Step S420: When data is contained in each first index subdirectory, calculate the hash value for each first index subdirectory;

[0089] Step S430: When no data is contained in each of the first index subdirectories, execute the root directory process.

[0090] In one example embodiment of this disclosure, after obtaining multiple first index directories corresponding to the index level through the above steps, it is possible to detect whether each first index subdirectory contains data. The root directory process is the process that indexes files starting from the root directory. Specifically, it is possible to detect whether each first index subdirectory contains data. When data is detected in a first index subdirectory, it indicates that the first index subdirectory contains a file. At this time, a hash value can be calculated for the first index subdirectory. If none of the first index subdirectories contain data, it indicates that the multiple first index directories corresponding to this index level do not contain the file to be indexed. In this case, it is necessary to return to the root directory for indexing, i.e., execute the root directory process. Specifically, the root directory process is the process that indexes files starting from the root directory.

[0091] It should be noted that this disclosure does not impose any special restrictions on the specific process of the root directory process.

[0092] Through the above steps S410 to S430, it can be detected whether each first index subdirectory contains data. If each first index subdirectory contains data, a hash value is calculated for each first index subdirectory; if each first index subdirectory does not contain data, the root directory process is executed. Through the embodiments of this disclosure, it is possible to detect whether the first index subdirectory contains data before calculating the hash value for each first index subdirectory. If it does not contain data, the root directory process is executed directly, improving indexing efficiency.

[0093] In one example embodiment of this disclosure, the root directory index level can be obtained, the root directory corresponding to the root directory index level can be obtained, a hash value can be calculated for each second index subdirectory, multiple files in the second index subdirectory can be determined based on the hash value, the metadata information corresponding to the multiple files can be called, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. (Refer to...) Figure 5 As shown, finding the file corresponding to multiple metadata information based on multiple metadata information may include the following steps S510 to S540:

[0094] Step S510: Obtain the root directory index level;

[0095] In one example embodiment of this disclosure, after determining the execution root directory process through the above steps, the root directory index level can be obtained. Specifically, the root directory refers to the top-level directory, and the root directory index level is the index level corresponding to the root directory, that is, the root directory index level is the highest index level.

[0096] It should be noted that this disclosure does not impose any special restrictions on the specific method for obtaining the root directory index level.

[0097] In one example embodiment of this disclosure, after obtaining the root directory index level, it can be determined whether the file to be indexed is in the root directory corresponding to the root directory index level. If the file to be indexed is in the root directory corresponding to the root directory index level, the following root directory process continues. If the file to be indexed is not in the root directory corresponding to the root directory index level, the indexing task fails.

[0098] In one example embodiment of this disclosure, after obtaining the root directory index level, it can be determined whether the index level of the file to be indexed is less than the root directory index level. If the index level of the file to be indexed is less than the root directory index level, the root directory process continues. If the index level of the file to be indexed is greater than or equal to the root directory index level, the indexing task fails.

[0099] Step S520: Obtain the root directory corresponding to the root directory index level; wherein, the root directory includes at least one second index subdirectory at the index sublevel, and each index sublevel includes at least one second index subdirectory.

[0100] In one example embodiment of this disclosure, after obtaining the root directory index level through the above steps, the root directory corresponding to the root directory index level can be obtained. The root directory includes at least one second index subdirectory at each index sublevel, and each index sublevel includes at least one second index subdirectory. Specifically, after obtaining the root directory index level, the root directory corresponding to the root directory index level can be found, and the root directory also includes multiple second index subdirectories.

[0101] Specifically, the root directory may also include multiple second index subdirectories at each index level, and all subdirectories at each index level are second index subdirectories.

[0102] It should be noted that this disclosure does not impose any special restrictions on the specific method for obtaining the root directory corresponding to the root directory index level.

[0103] Specifically, the second index subdirectory can be determined through recursive detection. Recursive detection refers to searching for or matching a specific element or pattern in a data structure. Recursive detection is usually implemented by recursively calling functions or methods. In each recursive call, an element or a child node in the current data structure is checked, and the result of the check determines whether to continue recursively.

[0104] In one example embodiment of this disclosure, after obtaining multiple second index subdirectories through the above steps, it is checked whether each second index subdirectory contains data. If each second index subdirectory contains data, the root directory process continues as described below. If each second index subdirectory does not contain data, the indexing task fails.

[0105] Step S530: Calculate the hash value for each second index subdirectory, and determine the multiple files in the second index subdirectory based on the hash value;

[0106] Step S540: Call the metadata information corresponding to multiple files, and find the file corresponding to the multiple metadata information based on the multiple metadata information.

[0107] In one example embodiment of this disclosure, after obtaining multiple second index subdirectories through the above steps, a hash value can be calculated for each second index subdirectory, and multiple files within the second index subdirectory can be determined based on the hash value. Specifically, when calculating the hash value for a second index subdirectory, the hash value can be calculated based on the data within the second index subdirectory. After obtaining the hash value of the second index subdirectory, multiple files within the second index subdirectory can be determined based on the hash value. This allows us to identify the files accessible by the second index subdirectory and retrieve the metadata information corresponding to the multiple files, thereby finding the files corresponding to the multiple metadata information.

[0108] It should be noted that this disclosure does not impose any special limitations on the specific method of determining multiple files in the second index subdirectory based on hash values.

[0109] In one example embodiment of this disclosure, after calculating the hash value for each second index subdirectory, metadata information corresponding to multiple files in each second index subdirectory can be called. If the call is successful, the file corresponding to the multiple metadata information is found based on the multiple metadata information; if the call fails, the indexing task fails.

[0110] Through the above steps S510-S540, the root directory index level can be obtained, the root directory corresponding to the root directory index level can be obtained, a hash value can be calculated for each second index subdirectory, multiple files in the second index subdirectory can be determined based on the hash value, the metadata information corresponding to the multiple files can be retrieved, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. Through the embodiments of this disclosure, when the first index directory corresponding to any index level does not contain the file to be indexed, the indexing can be performed by returning to the root directory, thereby improving the indexing efficiency.

[0111] In one example embodiment of this disclosure, file indexing requests are sent through different clients, each with a different index directory. Specifically, each client corresponds to a different index directory; that is, when performing an indexing task, different clients can index different index levels and the corresponding index directories for those levels, thus employing a hierarchical partitioning design. Through the scheme of this embodiment, the weight of each subtree can be balanced, achieving fast and scalable indexing.

[0112] like Figure 6 As shown, the index directories corresponding to client 1 are directory 4, directory 7, and directory 8, and the index directories corresponding to client 2 are root directory, directory 10, and directory 13. In this way, hierarchical partitioning is achieved.

[0113] In one exemplary embodiment of this disclosure, such as Figure 7 As shown, a file indexing method may include the following steps S701 to S7168:

[0114] Step S701: Obtain the index level; Step S702: Determine if the file to be indexed is in one of the multiple first index directories corresponding to the index level; if the file to be indexed is not in one of the multiple first index directories corresponding to the index level, execute the root directory process (step S709); Step S703: If the file to be indexed is in one of the multiple first index directories corresponding to the index level, determine if the index level of the file to be indexed is the same as the obtained index level; Step S704: If the same, create a data table in the first index directory; if different, execute the root directory process; Step S705: Obtain the multiple first index directories corresponding to the index level and check if each first index subdirectory contains data; if not, execute the root directory process; Step S706: If contained, calculate the hash value for each first index subdirectory; Step S707: Determine if determining multiple files in the first index subdirectory based on the hash value is successful; Step S708: If successful, call the metadata information corresponding to the multiple files; if unsuccessful, execute the root directory process.

[0115] Root directory process:

[0116] Step S710: Create a data table in the root directory; Step S711: Determine if the file to be indexed is in the root directory corresponding to the root directory index level; if the file to be indexed is not in the root directory corresponding to the index level, the indexing task fails (Step S712); Step S713: If the file to be indexed is in the root directory corresponding to the index level, determine if the index level of the file to be indexed is less than the obtained index level; if it is greater than or equal to, the indexing task fails; Step S714: If it is less than, check if each second index subdirectory contains data; if it does not contain data, the indexing task fails; Step S715: If it contains data, calculate the hash value for each second index subdirectory; Step S716: Determine if determining multiple files in the second index subdirectory based on the hash value is successful; Step S717: If successful, call the metadata information corresponding to multiple files; if unsuccessful, the indexing task fails.

[0117] Step S718: After obtaining the metadata information corresponding to multiple files, the database can be searched for the files corresponding to the metadata information of multiple files.

[0118] In one example embodiment of this disclosure, a file change log can be obtained, change-related information can be extracted from the file change log, the file identifier can be parsed into an absolute path, and it can be checked whether the file identifier is included in the cache. If the file identifier is not included in the cache, the file identifier is parsed to obtain the identifier path, and the identifier path is added to the suspicious file. If the file identifier is included in the cache, the absolute path is added to the suspicious file, thereby increasing the indexing frequency through the suspicious file. (Refer to...) Figure 8 As shown, when a file identifier is included in the cache, adding the absolute path to the suspicious file may include the following steps S810 to S840:

[0119] Step S810: Obtain the file change log and extract change-related information from the file change log;

[0120] In one example embodiment of this disclosure, a file change log can be obtained, and change-related information can be extracted from it. The file change log records file change events, and the change-related information includes the file identifier corresponding to the file. Specifically, the file change log can be used to record the file modification history. When a file is modified, its change log records the modification operations performed on the file, and change-related information can be extracted from the file change log, specifically, the file identifier corresponding to the file. Specifically, a file identifier (FID) is an identifier used in an operating system to uniquely identify a file. File identifiers are typically assigned by the file system; each file has a unique file identifier, which can be used to quickly locate and access files within the file system.

[0121] For example, change-related information may include file identifier, modification time, modifier, modification content, and file summaries before and after modification, in order to track and manage the file's change history.

[0122] It should be noted that this disclosure does not impose any special restrictions on the specific methods for extracting change-related information from the file change log.

[0123] Step S820: Resolve the file identifier into an absolute path and check whether the cache contains the file identifier;

[0124] For example, the fid2path (file identifier resolution) tool can be used to resolve file identifiers into absolute paths. Similarly, file identifiers can be resolved into absolute paths in an LRU (Least Recently Used) cache.

[0125] Step S830: If the file identifier is not included in the cache, parse the file identifier to obtain the identifier path, and add the identifier path to the suspicious file;

[0126] In step S840, when a file identifier is included in the cache, the absolute path is added to the suspicious file to increase the indexing frequency through the suspicious file.

[0127] In one example embodiment of this disclosure, after obtaining the file identifier corresponding to the file through the above steps, the file identifier can be parsed into an absolute path, and the cache can be checked to see if the file identifier is included. Specifically, if the cache does not include the file identifier, it means that the file corresponding to the file identifier may not be cached or has been changed. In this case, the file identifier can be parsed to obtain the identifier path, and the identifier path can be added to the suspicious file. If the cache includes the file identifier, the absolute path can be directly added to the suspicious file. When performing indexing tasks, the suspicious file can be referenced to improve the indexing frequency.

[0128] In one example embodiment of this disclosure, after the change events for multiple files have been processed, the change events for multiple files are cleared from the file change log. Clearing the change events from the file change log reduces the processing load on the file change log.

[0129] In one example embodiment of this disclosure, a file change log can be obtained through one thread, and change-related information can be extracted from the file change log through another thread.

[0130] Through the above steps S810 to S840, a file change log can be obtained, change-related information can be extracted from the file change log, the file identifier can be parsed into an absolute path, and it can be checked whether the cache includes the file identifier. If the cache does not include the file identifier, the file identifier is parsed to obtain the identifier path, and the identifier path is added to the suspicious file. If the cache includes the file identifier, the absolute path is added to the suspicious file to increase the indexing frequency through the suspicious file.

[0131] In one example embodiment of this disclosure, a file change log is obtained, and change-related information is extracted from the file change log. If the extraction is successful, the file identifier is parsed into an absolute path, and it is checked whether the file identifier is included in the cache. If the file identifier is not included in the cache, the file identifier is parsed to obtain an identifier path, and the identifier path is added to a suspicious file. If the file identifier is included in the cache, the absolute path is added to the suspicious file. If the extraction fails, the file identifier is parsed into an absolute path, and it is checked whether the file identifier is included in the cache. If the file identifier is not included in the cache, the file identifier is parsed to obtain an identifier path, the filename is deleted from the absolute path to obtain a directory name, and the identifier path is added to a suspicious file. If the file identifier is included in the cache, the absolute path is added to the suspicious file.

[0132] In one exemplary embodiment of this disclosure, such as Figure 9 As shown, a method for generating a suspicious file may include the following steps S901 to S918:

[0133] Step S901: Obtain the file change log and assign a change event to the value retrieved from the file change log; Step S902: Determine if the changed file exists; Step S903: Enter the log parsing process; Step S904: Extract change-related information from the change event; Step S905: Determine if the extraction process is normal; Step S906: If the extraction process is normal, parse the file identifier into an absolute path; Step S907: Check if the cache contains the file identifier; Step S908: If the cache contains the file identifier, add the absolute path to the suspicious file; Step S909: If the cache does not contain the file identifier, parse the file identifier to obtain the identifier path; Step S910: Add the identifier path to the... In the suspicious file; Step S911, if the extraction process is abnormal, parse the file identifier into an absolute path; Step S912, check if the cache includes the file identifier; Step S913, if the cache includes the file identifier, add the absolute path to the suspicious file; Step S914, if the cache does not include the file identifier, parse the file identifier to obtain the identifier path; Step S915, delete the file name from the absolute path to obtain the directory name; Step S916, add the identifier path to the suspicious file; Step S917, after the change events of multiple files are processed, clear the change events of multiple files in the file change log; Step S918, output the suspicious file.

[0134] In one exemplary embodiment of this disclosure, such as Figure 10 As shown, a re-indexer and an indexer can be configured in each storage system client. The re-indexer can extract change-related information from the file change log of the metadata server, parse the file identifier into an absolute path, check whether the cache includes the file identifier, and if the cache does not include the file identifier, parse the file identifier to obtain the identifier path and add the identifier path to the suspicious file. If the cache includes the file identifier, add the absolute path to the suspicious file to generate a suspicious file and send the generated suspicious file to the indexer. When performing indexing tasks, the indexer can refer to the suspicious file to increase the indexing frequency.

[0135] In the indexer, in response to a file indexing request, the index level can be obtained, multiple first index directories corresponding to the index level can be obtained, a hash value can be calculated for each first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, the metadata information corresponding to the multiple files can be called, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information.

[0136] In one exemplary embodiment of this disclosure, such as Figure 11As shown, a distributed storage system is provided, which includes a metadata server, a metadata index database, and a metadata index system set up on each storage system client.

[0137] The metadata server stores inode information and file change logs; the metadata index database includes multiple storage servers; the metadata index system includes an indexer and a crawler, with the indexer also containing a reindexer. The crawler can be used to crawl the entire file system.

[0138] In one embodiment of the file indexing method provided by this disclosure, in response to a file indexing request, an index level can be obtained, multiple first index directories corresponding to the index level can be obtained, a hash value can be calculated for each first index subdirectory, multiple files in the first index subdirectory can be determined based on the hash value, metadata information corresponding to the multiple files can be retrieved, and the files corresponding to the multiple metadata information can be found based on the multiple metadata information. Through the embodiments of this disclosure, when performing an indexing task, any index level can be obtained, and the metadata information of files can be searched in the directory corresponding to the index level, and the files can be found based on the metadata information, which can improve the efficiency of indexing and thus improve the efficiency of file management.

[0139] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0140] Furthermore, in an exemplary embodiment of this disclosure, a document indexing device is also provided. (Refer to...) Figure 12 As shown, a file indexing device 1200 includes: an index level acquisition module 1210, an index directory acquisition module 1220, a hash value calculation module 1230, and a metadata retrieval module 1240.

[0141] The system includes the following modules: an index level acquisition module, used to obtain the index level in response to a file indexing request; the index level can be any index level other than the index level corresponding to the root directory; an index directory acquisition module, used to obtain multiple first index directories corresponding to the index level; each first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; a hash value calculation module, used to calculate a hash value for each first index subdirectory, and determine multiple files in the first index subdirectory based on the hash value; and a metadata retrieval module, used to retrieve the metadata information corresponding to multiple files, and find the files corresponding to the multiple metadata information.

[0142] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, the apparatus further includes: a first determining unit, configured to determine whether the file to be indexed is in the multiple first index directories corresponding to the index level; a first processing unit, configured to obtain the multiple first index directories corresponding to the index level if the file to be indexed is in the multiple first index directories corresponding to the index level; and a second processing unit, configured to execute a root directory process if the file to be indexed is not in the multiple first index directories corresponding to the index level; wherein the root directory process is a process that indexes files starting from the root directory.

[0143] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, the apparatus further includes: a second determining unit for detecting whether each first index subdirectory contains data; a third processing unit for calculating hash values ​​for each first index subdirectory when data is contained in each first index subdirectory; and a fourth processing unit for executing a root directory process when no data is contained in each first index subdirectory; wherein the root directory process is a process that indexes files starting from the root directory.

[0144] In an exemplary embodiment of this disclosure, based on the aforementioned scheme, the root directory process includes: a root directory index level acquisition unit, used to acquire the root directory index level; wherein, the root directory index level is the index level corresponding to the root directory; a root directory acquisition unit, used to acquire the root directory corresponding to the root directory index level; wherein, the root directory includes at least one second index subdirectory at an index sublevel, and each index sublevel includes at least one second index subdirectory; a hash value calculation unit, used to calculate a hash value for each second index subdirectory, and determine multiple files in the second index subdirectory based on the hash value; and a metadata information acquisition unit, used to call the metadata information corresponding to multiple files, and find the files corresponding to the multiple metadata information based on the multiple metadata information.

[0145] In one exemplary embodiment of this disclosure, based on the foregoing scheme, file indexing requests are sent through different clients, each with a different index directory.

[0146] In an exemplary embodiment of this disclosure, based on the foregoing solution, the apparatus further includes: a file change log acquisition unit, configured to acquire a file change log and extract change-related information from the file change log; wherein the file change log is used to record file change events, and the change-related information includes a file identifier corresponding to the file; an absolute path parsing unit, configured to parse the file identifier into an absolute path and detect whether the cache includes the file identifier; a fifth processing unit, configured to parse the file identifier to obtain an identifier path when the cache does not include the file identifier, and add the identifier path to the suspicious file; and a sixth processing unit, configured to add the absolute path to the suspicious file when the cache includes the file identifier, so as to increase the indexing frequency through the suspicious file.

[0147] In one exemplary embodiment of this disclosure, based on the foregoing scheme, the file change log includes change events for multiple files, and the apparatus further includes: a change event clearing unit, used to clear the change events for multiple files in the file change log after the change events for multiple files have been processed.

[0148] Since the functional modules of the file indexing device in the example embodiments of this disclosure correspond to the steps of the example embodiments of the file indexing method described above, for details not disclosed in the device embodiments of this disclosure, please refer to the embodiments of the file indexing method described above.

[0149] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0150] Furthermore, in an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described file indexing method is also provided.

[0151] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be embodied in the following forms: a completely hardware embodiment, a completely software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0152] The following reference Figure 13 To describe an electronic device 1300 according to such an embodiment of the present disclosure. Figure 13The electronic device 1300 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0153] like Figure 13 As shown, the electronic device 1300 is manifested in the form of a general-purpose computing device. The components of the electronic device 1300 may include, but are not limited to: at least one processing unit 1310, at least one storage unit 1320, a bus 1330 connecting different system components (including storage unit 1320 and processing unit 1310), and a display unit 1340.

[0154] The storage unit stores program code, which can be executed by the processing unit 1310 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1310 can perform actions such as... Figure 2 Step S210, as shown, responds to a file indexing request and obtains an index level; wherein the file indexing request is used to indicate the file to be indexed, and the index level is any index level other than the index level corresponding to the root directory; Step S220, obtains multiple first index directories corresponding to the index level; wherein each first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; Step S230, calculates a hash value for each first index subdirectory, and determines multiple files in the first index subdirectory based on the hash value; Step S240, calls the metadata information corresponding to the multiple files, and searches for the files corresponding to the multiple metadata information based on the multiple metadata information.

[0155] Storage unit 1320 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1321 and / or cache memory 1322, and may further include read-only memory (ROM) 1323.

[0156] Storage unit 1320 may also include a program / utility 1324 having a set (at least one) program module 1325, such program module 1325 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0157] Bus 1330 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration open port, a processing unit, or a local bus using any of the various bus structures.

[0158] Electronic device 1300 can also communicate with one or more external devices 1370 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 1300, and / or with any device that enables electronic device 1300 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1350. Furthermore, electronic device 1300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1360. As shown, network adapter 1360 communicates with other modules of electronic device 1300 via bus 1330. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1300, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0159] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0160] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0161] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0163] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0164] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0165] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A file indexing method, characterized in that, The method includes: In response to a file indexing request, the index level is obtained; wherein, the index level is any index level other than the index level corresponding to the root directory; Obtain multiple first index directories corresponding to the index level; wherein, each first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; Calculate a hash value for each of the first index subdirectories, and determine multiple files in the first index subdirectories based on the hash values; The metadata information corresponding to the multiple files is retrieved, and the files corresponding to the multiple metadata information are found based on the multiple metadata information.

2. The method according to claim 1, characterized in that, The step of obtaining multiple first index directories corresponding to the index level includes: Determine whether the file to be indexed is in one of the multiple first index directories corresponding to the index level; If the file to be indexed is in multiple first index directories corresponding to the index level, then obtain the multiple first index directories corresponding to the index level; If the file to be indexed is not in any of the multiple first index directories corresponding to the index level, execute the root directory process; The root directory process is the process that indexes the files starting from the root directory.

3. The method according to claim 1, characterized in that, The calculation of hash values ​​for each of the first index subdirectories includes: Check whether each of the first index subdirectories contains data; When data is contained in each of the first index subdirectories, a hash value is calculated for each of the first index subdirectories; If none of the first indexed subdirectories contain data, execute the root directory process; The root directory process is the process that indexes the files starting from the root directory.

4. The method according to claim 2 or 3, characterized in that, The root directory process includes: Obtain the root directory index level; wherein, the root directory index level is the index level corresponding to the root directory; Obtain the root directory corresponding to the root directory index level; wherein, the root directory includes at least one second index subdirectory of the index sublevel, and each index sublevel includes at least one second index subdirectory; Calculate a hash value for each of the second index subdirectories, and determine multiple files in the second index subdirectories based on the hash values; The metadata information corresponding to the multiple files is retrieved, and the files corresponding to the multiple metadata information are found based on the multiple metadata information.

5. The method according to claim 1, characterized in that, The file indexing requests are sent through different clients, and each client corresponds to a different index directory.

6. The method according to claim 1, characterized in that, The method further includes: Obtain the file change log and extract change-related information from the file change log; wherein, the file change log is used to record file change events, and the change-related information includes the file identifier corresponding to the file; The file identifier is parsed into an absolute path, and the cache is checked to see if the file identifier is included. If the file identifier is not included in the cache, the file identifier is parsed to obtain the identifier path, and the identifier path is added to the suspicious file; When the file identifier is included in the cache, the absolute path is added to the suspicious file to increase the indexing frequency through the suspicious file.

7. The method according to claim 6, characterized in that, The file change log includes change events for multiple files, and the method further includes: After the change events for multiple files have been processed, the change events for multiple files are cleared from the file change log.

8. A document indexing device, characterized in that, The device includes: An index level acquisition module is used to acquire the index level in response to a file indexing request; wherein, the index level is any index level other than the index level corresponding to the root directory; An index directory acquisition module is used to acquire multiple first index directories corresponding to the index level; wherein, the first index directory includes at least one first index subdirectory of an index sublevel, and each index sublevel includes at least one first index subdirectory; The hash value calculation module is used to calculate a hash value for each of the first index subdirectories and determine multiple files in the first index subdirectories based on the hash value; The metadata retrieval module is used to retrieve the metadata information corresponding to the multiple files and to find the files corresponding to the multiple metadata information based on the multiple metadata information.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: One or more processors; as well as A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 7.

Images