A Retrieval Method and Device for an Object Storage File System

Abstract

The present invention discloses a retrieval method and apparatus for an object storage file system, relating to the field of computer technologies. A specific embodiment of the method includes: obtaining the directory tree node meta information passed by a hook function; wherein, the hook function is established in the object storage file system and is used to pass the changed directory tree node meta information when the directory tree node meta information of the object storage file system changes, and the directory tree node meta information includes a directory name, a file name, and a last modification time; storing the directory tree node meta information in an external index system and establishing a full-text index for all fields of the directory tree node meta information, and the external index system is independent of the object storage file system. This embodiment can solve the technical problems of large consumption of computing resources, inability to dynamically expand the index, and non-support for retrieval of non-indexed fields.

Description

Technical Field

[0001] The present invention relates to the field of computer technologies, and in particular, to a retrieval method and apparatus for an object storage file system. Background Art

[0002] In an object storage file system, file indexing is usually performed by building a B-Tree (multi-way search tree) or other data structures on the file system directory tree internally to establish an index. Changes to the file system directory tree will trigger real-time index updates. When a large number of files are written, index updates will consume a large amount of computing resources, significantly reducing the performance of the object storage system. In addition, the file system usually only builds indexes for preset fields internally, and cannot be changed or dynamically expanded after the index is established, and does not support retrieval for non-index fields. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a retrieval method and apparatus for an object storage file system to solve the technical problems of large consumption of computing resources, inability to dynamically expand indexes, and non-support for retrieval of non-index fields.

[0004] To achieve the above object, according to one aspect of the embodiments of the present invention, a retrieval method for an object storage file system is provided, including:

[0005] Obtaining metadata of a directory tree node passed by a hook function; wherein, the hook function is established in the object storage file system and is used to pass the changed metadata of the directory tree node when the metadata of the directory tree node in the object storage file system changes, and the metadata of the directory tree node includes a directory name, a file name, and a last modification time;

[0006] Storing the metadata of the directory tree node in an external index system and building a full-text index for all fields of the metadata of the directory tree node, where the external index system is independent of the object storage file system.

[0007] Optionally, obtaining the metadata of the directory tree node passed by the hook function includes:

[0008] Obtaining a message from a message queue and consuming the message to obtain metadata of a directory tree node; wherein, the hook function is used to asynchronously send the changed metadata of the directory tree node to the message queue when the metadata of the directory tree node changes.

[0009] Optionally, the metadata of the directory tree node further includes a file operation type, a file type, a file size, a file creator, a file creation time, and a file version number.

[0010] Optionally, after storing the directory tree node meta-information in an external index system and creating a full-text index for all fields of the directory tree node meta-information, the method further includes:

[0011] Receiving an index query request sent by a client;

[0012] Retrieving a retrieval result corresponding to the index query request from the external index system;

[0013] Returning the retrieval result to the client;

[0014] Wherein, each piece of content in the retrieval result includes directory tree node meta-information.

[0015] In addition, according to another aspect of the embodiments of the present invention, a retrieval method for an object storage file system is provided, including:

[0016] Creating a hook function in the object storage file system, where the hook function is used to transmit the changed directory tree node meta-information when the directory tree node meta-information of the object storage file system changes, and the directory tree node meta-information includes a directory name, a file name, and a last modification time;

[0017] Obtaining the directory tree node meta-information transmitted by the hook function;

[0018] Storing the directory tree node meta-information in an external index system and creating a full-text index for all fields of the directory tree node meta-information, where the external index system is independent of the object storage file system.

[0019] Optionally, the hook function is used to asynchronously send the changed directory tree node meta-information to a message queue when the directory tree node meta-information changes.

[0020] Optionally, obtaining the directory tree node meta-information transmitted by the hook function includes:

[0021] Obtaining a message from the message queue and consuming the message to obtain the directory tree node meta-information.

[0022] Optionally, the directory tree node meta-information further includes a file operation type, a file type, a file size, a file creator, a file creation time, and a file version number.

[0023] Optionally, after storing the directory tree node meta-information in an external index system and creating a full-text index for all fields of the directory tree node meta-information, the method further includes:

[0024] Receiving an index query request sent by a client;

[0025] Retrieve the retrieval result corresponding to the index query request from the external index system;

[0026] Return the retrieval result to the client;

[0027] Wherein, each piece of content in the retrieval result includes directory tree node meta information.

[0028] Optionally, after returning the retrieval result to the client, it further includes:

[0029] Receive a file read / write request sent by the client; wherein, the file read / write request carries a directory name and a file name;

[0030] Read the target file corresponding to the file read / write request from the object storage file system;

[0031] Return the target file and the global tag of the target file to the client.

[0032] Optionally, the global tag is the last modification time or the file version number.

[0033] In addition, according to another aspect of the embodiments of the present invention, a retrieval device for an object storage file system is provided, including:

[0034] An acquisition module, configured to acquire directory tree node meta information passed by a hook function; wherein, the hook function is established in the object storage file system and is used to pass the changed directory tree node meta information when the directory tree node meta information in the object storage file system changes, and the directory tree node meta information includes a directory name, a file name, and a last modification time;

[0035] A storage module, configured to store the directory tree node meta information in an external index system and establish a full-text index for all fields of the directory tree node meta information, and the external index system is independent of the object storage file system.

[0036] Optionally, the acquisition module is further configured to:

[0037] Acquire a message from a message queue and consume the message to obtain directory tree node meta information; wherein, the hook function is used to asynchronously send the changed directory tree node meta information to the message queue when the directory tree node meta information changes.

[0038] Optionally, the directory tree node meta information further includes a file operation type, a file type, a file size, a file creator, a file creation time, and a file version number.

[0039] Optionally, it further includes a retrieval module, configured to:

[0040] Receive an index query request sent by a client;

[0041] Retrieve a retrieval result corresponding to the index query request from the external index system;

[0042] Return the retrieval result to the client;

[0043] Wherein, each piece of content in the retrieval result includes directory tree node meta-information.

[0044] In addition, according to another aspect of the embodiments of the present invention, a retrieval device for an object storage file system is provided, including:

[0045] A creation module, configured to create a hook function in the object storage file system, where the hook function is used to transfer the changed directory tree node meta-information when the directory tree node meta-information in the object storage file system changes, and the directory tree node meta-information includes a directory name, a file name, and a last modification time;

[0046] An acquisition module, configured to acquire the directory tree node meta-information transferred by the hook function;

[0047] A storage module, configured to store the directory tree node meta-information in an external index system and create a full-text index for all fields of the directory tree node meta-information, where the external index system is independent of the object storage file system.

[0048] Optionally, the hook function is used to asynchronously send the changed directory tree node meta-information to a message queue when the directory tree node meta-information changes.

[0049] Optionally, the acquisition module is further configured to:

[0050] Acquire a message from the message queue and consume the message, so as to obtain directory tree node meta-information.

[0051] Optionally, the directory tree node meta-information further includes a file operation type, a file type, a file size, a file creator, a file creation time, and a file version number.

[0052] Optionally, it further includes a retrieval module, configured to:

[0053] Receive an index query request sent by a client;

[0054] Retrieve a retrieval result corresponding to the index query request from the external index system;

[0055] Return the retrieval result to the client;

[0056] Among them, each piece of content in the retrieval result includes directory tree node meta information.

[0057] Optionally, it further includes a read / write module for:

[0058] Receiving a file read / write request sent by the client; wherein, the file read / write request carries a directory name and a file name;

[0059] Reading a target file corresponding to the file read / write request from the object storage file system;

[0060] Returning the target file and the global tag of the target file to the client.

[0061] Optionally, the global tag is the last modification time or the file version number.

[0062] According to another aspect of the embodiments of the present invention, an electronic device is further provided, including:

[0063] One or more processors;

[0064] A storage device for storing one or more programs,

[0065] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0066] According to another aspect of the embodiments of the present invention, a computer-readable medium is further provided, on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0067] One of the embodiments of the above invention has the following advantages or beneficial effects: Because the technical means of obtaining the directory tree node meta information passed by the hook function, storing the directory tree node meta information in an external index system independent of the object storage file system and establishing a full-text index for all fields in the directory tree node meta information are adopted, the technical problems of large consumption of computing resources, inability to dynamically expand the index, and non-support for non-indexed field retrieval in the prior art are overcome. When the directory tree node meta information of the object storage file system changes, the embodiments of the present invention pass the changed directory tree node meta information through the hook function, and do not directly update the index inside the object storage file system, but store the directory tree node meta information in the external index system and establish a full-text index for the directory tree node meta information, which can not only avoid affecting the performance of the object storage file system when a large amount of data changes, but also perform information retrieval and fuzzy query in any dimension in the external index system.

[0068] The further effects of the above non-conventional optional methods will be described in combination with specific embodiments below. Brief Description of the Drawings

[0069] The drawings are used to better understand the present invention and do not unduly limit the present invention. Among them:

[0070] Figure 1 is a schematic diagram of the main process of a retrieval method for an object storage file system according to an embodiment of the present invention;

[0071] Figure 2 is a schematic diagram of building a full-text index according to an embodiment of the present invention;

[0072] Figure 3 is a schematic diagram of the main process of a retrieval method for an object storage file system according to a referenceable embodiment of the present invention;

[0073] Figure 4 is a schematic diagram of retrieving a file from an object storage file system according to an embodiment of the present invention;

[0074] Figure 5 is a schematic diagram of building an index and retrieving a file according to an embodiment of the present invention;

[0075] Figure 6 is a schematic diagram of the main process of a retrieval method for an object storage file system according to another embodiment of the present invention;

[0076] Figure 7 is a schematic diagram of the main process of a retrieval method for an object storage file system according to another referenceable embodiment of the present invention;

[0077] Figure 8 is a schematic diagram of the main modules of a retrieval device for an object storage file system according to an embodiment of the present invention;

[0078] Figure 9 is a schematic diagram of the main modules of a retrieval device for an object storage file system according to another embodiment of the present invention;

[0079] Figure 10 is an exemplary system architecture diagram to which an embodiment of the present invention can be applied;

[0080] Figure 11 is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. Detailed Description of the Embodiments

[0081] The exemplary embodiments of the present invention will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present invention are included to assist understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0082] Figure 1 It is a schematic diagram of the main process of the retrieval method of the object storage file system according to an embodiment of the present invention. As an embodiment of the present invention, as Figure 1 shown, the retrieval method of the object storage file system may include:

[0083] Step 101, obtain the directory tree node meta information passed by the hook function; wherein, the hook function is established in the object storage file system and is used to pass the changed directory tree node meta information when the directory tree node meta information of the object storage file system changes, and the directory tree node meta information includes the directory name, file name, and last modification time.

[0084] In order to be able to pass the changed directory tree node meta information when the directory tree node meta information of the object storage file system changes, first, a hook function T needs to be established in the object storage file system, and a directory tree node meta information change event is constructed through this hook function. When the client performs create, modify, or delete operations on files and directories in the object storage file system through the object storage interface, the object storage file system calls the hook function at the directory tree change point and passes the directory tree node meta information as a parameter through this hook function. It should be noted that after the hook function is called, it does not block and returns immediately, so it will not affect the normal change process of the directory tree in the object storage file system.

[0085] Optionally, the directory tree node meta information includes necessary information such as the directory name, file name, and last modification time. Files and directories can be accurately located through the directory tree node meta information. Optionally, the directory tree node meta information may further include the file operation type, file type, file size, file creator, file creation time, and file version number. Passing these information helps the external indexing system build a full-text index, so that the client can perform full-text retrieval in the external indexing system through these information. It should be pointed out that whenever a file is modified, the file version number increases sequentially.

[0086] Optionally, the hook function is used to asynchronously send the changed directory tree node meta information to the message queue when the directory tree node meta information changes. As Figure 2As shown, when the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node meta-information), caches it in the server internal queue, and sends the event to the message queue asynchronously.

[0087] When the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the object storage file system calls the hook function at the directory tree change, and passes the directory tree node meta-information as a parameter through the hook function, so as to obtain the directory tree node meta-information.

[0088] Optionally, step 101 may include: obtaining a message from the message queue and consuming the message to obtain the directory tree node meta-information. As Figure 2 As shown, when the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node meta-information), sends the event to the message queue asynchronously, and then the external indexing system obtains the message from the message queue and consumes the message to obtain the directory tree node meta-information.

[0089] Step 102, storing the directory tree node meta-information in the external indexing system and establishing a full-text index for all fields of the directory tree node meta-information, where the external indexing system is independent of the object storage file system.

[0090] The obtained directory tree node meta-information is stored in the external indexing system, and the external indexing system establishes a full-text index for all fields of the directory tree node meta-information, rather than establishing an index for specific fields, to support information retrieval and fuzzy retrieval in any dimension.

[0091] Therefore, when the client retrieves an object storage file, it first retrieves the directory tree node meta-information from the external indexing system, and then locates the target file in the object storage file system through the directory tree node meta-information, so as to perform file operations on it, such as creating, modifying, or deleting file operations.

[0092] Optionally, the last modification time or the file version number may be used as a global tag. In the embodiments of the present invention, the global tag is saved in each directory tree node meta-information to determine whether the node is in the latest state, and the global tag is returned to the client together in the retrieval result to facilitate the client to handle the indexing system delay.

[0093] According to the various embodiments described above, it can be seen that the embodiments of the present invention solve the technical problems of large consumption of computing resources, inability to dynamically expand the index, and non-support for retrieval of non-indexed fields in the prior art by means of obtaining the meta-information of directory tree nodes transmitted by the hook function, storing the meta-information of directory tree nodes in an external index system independent of the object storage file system, and establishing a full-text index for all fields in the meta-information of directory tree nodes. When the meta-information of directory tree nodes in the object storage file system changes, the embodiments of the present invention transmit the changed meta-information of directory tree nodes through the hook function, and do not directly update the index inside the object storage file system. Instead, the meta-information of directory tree nodes is stored in the external index system and a full-text index is established for the meta-information of directory tree nodes, which can not only avoid affecting the performance of the object storage file system during large-scale data changes, but also perform information retrieval and fuzzy retrieval in any dimension in the external index system.

[0094] Figure 3 It is a schematic diagram of the main process of the retrieval method of the object storage file system according to a reference embodiment of the present invention. As another embodiment of the present invention, as Figure 3 shown, the retrieval method of the object storage file system may include:

[0095] Step 301, obtaining the meta-information of directory tree nodes transmitted by the hook function; wherein, the hook function is established in the object storage file system and is used to transmit the changed meta-information of directory tree nodes when the meta-information of directory tree nodes in the object storage file system changes, and the meta-information of directory tree nodes includes directory name, file name, and last modification time.

[0096] Step 302, storing the meta-information of directory tree nodes in the external index system and establishing a full-text index for all fields of the meta-information of directory tree nodes, and the external index system is independent of the object storage file system.

[0097] Step 303, receiving an index query request sent by the client.

[0098] After establishing a full-text index in the external index system, as Figure 4 shown, the client can perform information retrieval and fuzzy retrieval in any dimension such as directory name, file name, last modification time, file operation type, file type, file size, file creator, file creation time, and file version number through the interface provided by the external index system.

[0099] Step 304, retrieving the retrieval result corresponding to the index query request from the external index system. Each item in the retrieval result includes the meta-information of directory tree nodes.

[0100] The index query request carries the retrieval fields input by the user through the client. The external index system performs full-text retrieval based on the retrieval fields to obtain the retrieval results corresponding to the index query request. Each piece of content in the retrieval results includes directory tree node meta-information.

[0101] Step 305: Return the retrieval results to the client.

[0102] The external index system returns the retrieval results that meet the query conditions to the client. Each piece of content in the retrieval result list includes directory tree node meta-information, such as directory name, file name, last modification time, file operation type, file type, file size, file creator, file creation time, and file version number, etc.

[0103] It should be noted that in the retrieval results returned by the external index system, the last modification time or file version number in the directory tree node meta-information can be used as a global tag to facilitate the client to handle the index system delay.

[0104] The client user views the retrieval results returned by the external index system and sends a file read / write request to the object storage file system according to the retrieval results. The object storage file system receives the file read / write request sent by the client. In order to locate the target file in the object storage file system, the file read / write request carries the directory name and file name. As Figure 4 shown, the object storage file system locates the target file according to the directory name and file name carried in the file read / write request. The object storage file system returns the located target file to the client. At the same time, the target file carries a global tag, which can be the last modification time or file version number. The client compares the global tag returned by the external index system with the global tag returned by the object storage file system to find out whether there is an external index system delay by comparison.

[0105] The client can perform related operations on the target file according to different business requirements, such as reading, creating, modifying, or deleting, etc. When the client performs create, modify, or delete operations on files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node meta-information), caches it in the server internal queue, and sends the event to the message queue asynchronously. Then the external index system obtains the message from the message queue and consumes the message to obtain the directory tree node meta-information, and then stores the directory tree node meta-information in the external index system, as Figure 5 shown.

[0106] In addition, the specific implementation content of the retrieval method of the object storage file system in a referenceable embodiment of the present invention has been described in detail in the above-mentioned retrieval method of the object storage file system, so the repeated content will not be described here.

[0107] Figure 6 is a schematic diagram of the main process of the retrieval method of an object storage file system according to another embodiment of the present invention. As another embodiment of the present invention, as Figure 6 shown, the retrieval method of the object storage file system may include:

[0108] Step 601, establish a hook function in the object storage file system, where the hook function is used to transmit the changed directory tree node meta information when the meta information of the directory tree node of the object storage file system changes, and the directory tree node meta information includes directory name, file name, and last modification time.

[0109] In order to be able to transmit the changed directory tree node meta information when the meta information of the directory tree node of the object storage file system changes, first, a hook function T needs to be established in the object storage file system, and a directory tree node meta information change event is constructed through this hook function. When the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the object storage file system calls the hook function at the directory tree change, and transmits the directory tree node meta information through this hook function. It should be noted that after the hook function is called, it does not block and returns immediately, so it will not affect the normal change process of the directory tree in the object storage file system.

[0110] Optionally, the directory tree node meta information includes necessary information such as directory name, file name, and last modification time. Files and directories can be accurately located through the directory tree node meta information. Optionally, the directory tree node meta information may further include file operation type, file type, file size, file creator, file creation time, and file version number. Transmitting these information helps the external index system build a full-text index, so as to facilitate the client to perform full-text retrieval in the external index system through these information. It should be pointed out that whenever a file is modified, the file version number increases sequentially.

[0111] Optionally, the hook function is used to asynchronously send the changed directory tree node meta information to the message queue when the directory tree node meta information changes. As Figure 2 shown, when the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node meta information), caches it in the server internal queue, and asynchronously sends the event to the message queue.

[0112] Step 602, obtain the directory tree node meta information passed by the hook function.

[0113] When the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the object storage file system calls a hook function at the directory tree change point and passes the metadata of the directory tree node through this hook function to obtain the metadata of the directory tree node.

[0114] Optionally, step 602 may include: obtaining a message from the message queue and consuming the message to obtain the metadata of the directory tree node. As Figure 2 shown, when the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (metadata of the directory tree node), sends the event to the message queue asynchronously, and then the external indexing system obtains the message from the message queue and consumes the message to obtain the metadata of the directory tree node.

[0115] Step 603, storing the metadata of the directory tree node in the external indexing system and creating a full-text index for all fields of the metadata of the directory tree node, where the external indexing system is independent of the object storage file system.

[0116] The obtained metadata of the directory tree node is stored in the external indexing system, and the external indexing system creates a full-text index for all fields of the metadata of the directory tree node, rather than creating an index for specific fields, to support information retrieval and fuzzy retrieval in any dimension.

[0117] Therefore, when the client retrieves an object storage file, it first retrieves the metadata of the directory tree node from the external indexing system, and then locates the target file in the object storage file system through the metadata of the directory tree node, so as to perform file operations on it, such as creating, modifying, or deleting file operations.

[0118] Optionally, the last modification time or the file version number may be used as a global tag. In the embodiments of the present invention, the global tag is saved in each metadata of the directory tree node to determine whether the node is in the latest state, and the global tag is returned to the client together in the retrieval result to facilitate the client to handle the indexing system delay.

[0119] According to the various embodiments described above, it can be seen that in the embodiments of the present invention, by obtaining the directory tree node meta-information passed by the hook function, storing the directory tree node meta-information in an external indexing system independent of the object storage file system, and establishing a full-text index for all fields in the directory tree node meta-information, the technical problems in the prior art of high computational resource consumption, inability to dynamically expand the index, and non-support for retrieving non-indexed fields are solved. When the directory tree node meta-information of the object storage file system changes, the embodiments of the present invention pass the changed directory tree node meta-information through the hook function, and do not directly update the index inside the object storage file system. Instead, the directory tree node meta-information is stored in the external indexing system and a full-text index is established for the directory tree node meta-information, which can not only avoid affecting the performance of the object storage file system when a large amount of data changes, but also perform information retrieval and fuzzy retrieval in any dimension in the external indexing system.

[0120] Figure 7 is a schematic diagram of the main process of the retrieval method of the object storage file system according to another referenceable embodiment of the present invention. As another embodiment of the present invention, as Figure 7 shown, the retrieval method of the object storage file system may include:

[0121] Step 701, establish a hook function in the object storage file system, where the hook function is used to asynchronously send the changed directory tree node meta-information to the message queue when the directory tree node meta-information changes.

[0122] In order to be able to pass the changed directory tree node meta-information when the directory tree node meta-information of the object storage file system changes, first, a hook function T needs to be established in the object storage file system. Through this hook function, a directory tree node meta-information change event is constructed. When the client performs create, modify, or delete operations on files and directories in the object storage file system through the object storage interface, the object storage file system calls the hook function at the directory tree change, passes the directory tree node meta-information through this hook function, and sends the event to the message queue asynchronously. Among them, the directory tree node meta-information includes directory name, file name, last modification time, file operation type, file type, file size, file creator, file creation time, and file version number.

[0123] Step 702, obtain the message from the message queue and consume the message to obtain the directory tree node meta-information.

[0124] When the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node metadata), sends the event to the message queue asynchronously, and then the external indexing system fetches and consumes the message from the message queue to obtain the directory tree node metadata.

[0125] Step 703: Store the directory tree node metadata in the external indexing system and create a full-text index for all fields of the directory tree node metadata. The external indexing system is independent of the object storage file system and is used to create a full-text index for all fields.

[0126] The obtained directory tree node metadata is stored in the external indexing system, and the external indexing system creates a full-text index for all fields of the directory tree node metadata instead of specific fields to support information retrieval and fuzzy retrieval in any dimension.

[0127] Step 704: Receive the index query request sent by the client.

[0128] Through the interface provided by the external indexing system, the client can perform information retrieval and fuzzy retrieval in any dimension such as directory name, file name, last modification time, file operation type, file type, file size, file creator, file creation time, and file version number.

[0129] Step 705: Retrieve the retrieval result corresponding to the index query request from the external indexing system and return the retrieval result to the client.

[0130] The index query request carries the retrieval fields input by the user through the client. The external indexing system performs a full-text search based on the retrieval fields to obtain the retrieval result corresponding to the index query request. Each item in the retrieval result includes the directory tree node metadata.

[0131] The external indexing system returns the retrieval results that meet the query conditions to the client. Each item in the retrieval result list includes the directory tree node metadata, such as directory name, file name, last modification time, file operation type, file type, file size, file creator, file creation time, and file version number.

[0132] It should be noted that in the retrieval results returned by the external indexing system, the last modification time or file version number in the directory tree node metadata can be used as a global tag to facilitate the client to handle the indexing system delay.

[0133] Step 706: Receive the file read / write request sent by the client.

[0134] The client user views the retrieval results returned by the external index system and sends a file read / write request to the object storage file system according to the retrieval results. The object storage file system receives the file read / write request sent by the client. In order to locate the target file in the object storage file system, the file read / write request carries a directory name and a file name.

[0135] Step 707, read the target file corresponding to the file read / write request from the object storage file system.

[0136] Such as Figure 4 shown, the object storage file system locates the target file according to the directory name and file name carried in the file read / write request.

[0137] Step 708, return the target file and the global label of the target file (which can be the last modification time or the file version number) to the client.

[0138] The object storage file system returns the located target file to the client. At the same time, the target file carries a global label, which can be the last modification time or the file version number. The client compares the global label returned by the external index system with the global label returned by the object storage file system to find out whether there is a delay in the external index system through comparison.

[0139] The client can perform related operations on the target file according to different business requirements, such as reading, creating, modifying, or deleting, etc. When the client creates, modifies, or deletes files and directories in the object storage file system through the object storage interface, the hook function triggers a change event (directory tree node meta-information), caches it in the server internal queue, and sends the event to the message queue asynchronously. Then the external index system obtains the message from the message queue and consumes the message to obtain the directory tree node meta-information, and then stores the directory tree node meta-information in the external index system, as Figure 5 shown.

[0140] In addition, the specific implementation content of the retrieval method of the object storage file system in another referenceable embodiment of the present invention has been described in detail in the above-mentioned retrieval method of the object storage file system, so the repeated content will not be described here.

[0141] Figure 8 is a schematic diagram of the main modules of the retrieval device of the object storage file system according to the embodiment of the present invention. Such as Figure 8As shown in the figure, the retrieval device 800 of the object storage file system includes an acquisition module 801 and a storage module 802. Among them, the acquisition module 801 is used to acquire the directory tree node meta-information passed by the hook function. Among them, the hook function is established in the object storage file system and is used to transmit the changed directory tree node meta-information when the directory tree node meta-information of the object storage file system changes. The directory tree node meta-information includes directory name, file name, and last modification time. The storage module 802 is used to store the directory tree node meta-information in an external index system and create a full-text index for all fields of the directory tree node meta-information. The external index system is independent of the object storage file system.

[0142] Optionally, the acquisition module 801 is further used for:

[0143] Obtain messages from the message queue and consume the messages to obtain the directory tree node meta-information. Among them, the hook function is used to asynchronously send the changed directory tree node meta-information to the message queue when the directory tree node meta-information changes.

[0144] Optionally, the directory tree node meta-information further includes file operation type, file type, file size, file creator, file creation time, and file version number.

[0145] Optionally, it further includes a retrieval module for:

[0146] Receive an index query request sent by the client;

[0147] Retrieve the retrieval result corresponding to the index query request from the external index system;

[0148] Return the retrieval result to the client;

[0149] Among them, each content in the retrieval result includes the directory tree node meta-information.

[0150] It should be noted that the specific implementation content of the retrieval device of the object storage file system in the present invention has been described in detail in the above-mentioned retrieval method of the object storage file system, so the repeated content will not be described here.

[0151] Figure 9 is a schematic diagram of the main modules of the retrieval device of the object storage file system according to another embodiment of the present invention. As Figure 9As shown in the figure, the retrieval device 900 of the object storage file system includes a creation module 901, an acquisition module 902, and a storage module 903. Among them, when the meta-information of the directory tree node of the object storage file system changes, the creation module 901 transmits the changed meta-information of the directory tree node. The meta-information of the directory tree node includes directory name, file name, and last modification time. The acquisition module 902 is used to acquire the meta-information of the directory tree node transmitted by the hook function. The storage module 903 is used to store the meta-information of the directory tree node in an external index system and create a full-text index for all fields of the meta-information of the directory tree node. The external index system is independent of the object storage file system.

[0152] Optionally, the hook function is used to asynchronously send the changed meta-information of the directory tree node to a message queue when the meta-information of the directory tree node changes.

[0153] Optionally, the acquisition module 902 is further used to:

[0154] Obtain a message from the message queue and consume the message to obtain the meta-information of the directory tree node.

[0155] Optionally, the meta-information of the directory tree node further includes file operation type, file type, file size, file creator, file creation time, and file version number.

[0156] Optionally, it further includes a retrieval module, which is used to:

[0157] Receive an index query request sent by a client;

[0158] Retrieve a retrieval result corresponding to the index query request from the external index system;

[0159] Return the retrieval result to the client;

[0160] Among them, each item of content in the retrieval result includes the meta-information of the directory tree node.

[0161] Optionally, it further includes a read-write module, which is used to:

[0162] Receive a file read-write request sent by the client. Among them, the file read-write request carries a directory name and a file name;

[0163] Read a target file corresponding to the file read-write request from the object storage file system;

[0164] Return the target file and the global label of the target file to the client.

[0165] Optionally, the global label is the last modification time or the file version number.

[0166] It should be noted that the specific implementation content of the retrieval device of the object storage file system described in the present invention has been described in detail in the above-mentioned retrieval method of the object storage file system, so the repeated content will not be described here.

[0167] Figure 10 An exemplary system architecture 1000 is shown to which the retrieval method of the object storage file system or the retrieval device of the object storage file system according to the embodiments of the present invention can be applied.

[0168] As Figure 10 shown, the system architecture 1000 may include terminal devices 1001, 1002, 1003, a network 1004, and a server 1005. The network 1004 is used to provide a medium for communication links between the terminal devices 1001, 1002, 1003 and the server 1005. The network 1004 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0169] Users can use the terminal devices 1001, 1002, 1003 to interact with the server 1005 through the network 1004 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 1001, 1002, 1003, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0170] The terminal devices 1001, 1002, 1003 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0171] The server 1005 may be a server providing various services, such as a background management server (only as an example) that supports the shopping websites browsed by users using the terminal devices 1001, 1002, 1003. The background management server may analyze and process data such as item information query requests received, and feedback the processing results to the terminal devices.

[0172] It should be noted that the retrieval method of the object storage file system provided by the embodiments of the present invention is generally executed by the server 1005. Correspondingly, the retrieval device of the object storage file system is generally set in the server 1005.

[0173] It should be understood that Figure 10 the numbers of the terminal devices, the network, and the server in

[0174] are merely illustrative. According to the implementation requirements, there may be any number of terminal devices, networks, and servers. Figure 11, which shows a schematic structural diagram of a computer system 1100 of a terminal device suitable for implementing the embodiments of the present invention. Figure 11 The terminal device shown is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0175] As Figure 11 shown, the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1102 or the program loaded from the storage section 1108 into the random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the system 1100 are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. The input / output (I / O) interface 1105 is also connected to the bus 1104.

[0176] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as required. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as required so that a computer program read from it can be installed into the storage section 1108 as required.

[0177] Specifically, according to the embodiments disclosed in the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present invention include a computer program, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication section 1109, and / or installed from the removable medium 1111. When the computer program is executed by the central processing unit (CPU) 1101, the above functions defined in the system of the present invention are executed.

[0178] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer programs according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments of the present invention can be implemented in software or in hardware. The described modules can also be provided in a processor. For example, it can be described as: a processor includes a creation module, an acquisition module, and a storage module. In some cases, the names of these modules do not constitute a limitation on the modules themselves.

[0181] As another aspect, the present invention further provides a computer-readable medium. The computer-readable medium can be included in the device described in the above embodiments; or it can exist alone without being assembled into the device. The above computer-readable medium carries one or more programs. When the one or more programs are executed by the device, the device implements the following method: creating a hook function in an object storage file system, where the hook function is used to transmit the changed meta-information of the directory tree node when the meta-information of the directory tree node of the object storage file system changes. The meta-information of the directory tree node includes a directory name, a file name, and a last modification time; acquiring the meta-information of the directory tree node transmitted by the hook function; storing the meta-information of the directory tree node in an external indexing system and creating a full-text index for all fields of the meta-information of the directory tree node, where the external indexing system is independent of the object storage file system.

[0182] According to the technical solution of the embodiments of the present invention, by adopting the technical means of acquiring the meta-information of the directory tree node transmitted by the hook function, storing the meta-information of the directory tree node in an external indexing system independent of the object storage file system, and creating a full-text index for all fields in the meta-information of the directory tree node, the technical problems of high computing resource consumption, inability to dynamically expand the index, and non-support for non-indexed field retrieval in the prior art are overcome. When the meta-information of the directory tree node of the object storage file system changes, the embodiments of the present invention transmit the changed meta-information of the directory tree node through the hook function, and do not directly update the index inside the object storage file system. Instead, the meta-information of the directory tree node is stored in the external indexing system and a full-text index is created for the meta-information of the directory tree node, which can not only avoid affecting the performance of the object storage file system during large-scale data changes, but also perform information retrieval and fuzzy queries in any dimension in the external indexing system.

[0183] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A retrieval method for an object storage file system, characterized in that including: Obtain the metadata of the directory tree node passed by the hook function asynchronously; wherein, the hook function is established in the object storage file system and is used to pass the changed metadata of the directory tree node when the metadata of the directory tree node in the object storage file system changes, and the metadata of the directory tree node includes directory name, file name, and last modification time; Store the metadata of the directory tree node in an external index system and create a full-text index for all fields of the metadata of the directory tree node, and the external index system is independent of the object storage file system.

2. The method according to claim 1, characterized in that Obtaining the metadata of the directory tree node passed by the hook function includes: Obtain a message from the message queue and consume the message to obtain the metadata of the directory tree node; wherein, the hook function is used to asynchronously send the changed metadata of the directory tree node to the message queue when the metadata of the directory tree node changes.

3. The method according to claim 1, characterized in that The metadata of the directory tree node further includes file operation type, file type, file size, file creator, file creation time, and file version number.

4. The method according to claim 1, wherein After storing the metadata of the directory tree node in the external index system and creating a full-text index for all fields of the metadata of the directory tree node, it further includes: Receive an index query request sent by the client; Retrieve the retrieval result corresponding to the index query request from the external index system; Return the retrieval result to the client; wherein, each content in the retrieval result includes the metadata of the directory tree node.

5. A retrieval method for an object storage file system, characterized in that, including: Establish a hook function in the object storage file system, and the hook function is used to asynchronously pass the changed metadata of the directory tree node when the metadata of the directory tree node in the object storage file system changes, and the metadata of the directory tree node includes directory name, file name, and last modification time; Obtain the metadata of the directory tree node passed by the hook function; Store the metadata of the directory tree node in an external index system and create a full-text index for all fields of the metadata of the directory tree node, and the external index system is independent of the object storage file system.

6. The method according to claim 5, wherein The hook function is used to asynchronously send the changed metadata of the directory tree node to the message queue when the metadata of the directory tree node changes.

7. The method according to claim 6, wherein Obtaining the metadata of the directory tree node passed by the hook function includes: Obtain a message from the message queue and consume the message to obtain the metadata of the directory tree node.

8. The method according to claim 5, characterized in that, The metadata of the directory tree node further includes file operation type, file type, file size, file creator, file creation time, and file version number.

9. The method according to claim 5, wherein After storing the metadata of the directory tree node in the external index system and creating a full-text index for all fields of the metadata of the directory tree node, it further includes: Receive an index query request sent by the client; Retrieve the retrieval result corresponding to the index query request from the external index system; Return the retrieval result to the client; wherein, each content in the retrieval result includes the metadata of the directory tree node; Receive a file read / write request sent by the client; wherein, the file read / write request carries a directory name and a file name; Read the target file corresponding to the file read / write request from the object storage file system; Return the target file and the global tag of the target file to the client.

10. The method according to claim 9, wherein The global tag is the last modification time or the file version number.

11. A retrieval device for an object storage file system, characterized in that, It includes: An acquisition module, configured to acquire the directory tree node meta-information passed by the hook function in an asynchronous manner; wherein, the hook function is established in the object storage file system and is used to pass the changed directory tree node meta-information when the directory tree node meta-information of the object storage file system changes, and the directory tree node meta-information includes the directory name, file name, and last modification time; A storage module, configured to store the directory tree node meta-information in an external index system and establish a full-text index for all fields of the directory tree node meta-information, and the external index system is independent of the object storage file system.

12. A retrieval device for an object storage file system, characterized in that, It includes: An establishment module, configured to establish a hook function in the object storage file system, and the hook function is used to pass the changed directory tree node meta-information in an asynchronous manner when the directory tree node meta-information of the object storage file system changes, and the directory tree node meta-information includes the directory name, file name, and last modification time; An acquisition module, configured to acquire the directory tree node meta-information passed by the hook function; A storage module, configured to store the directory tree node meta-information in an external index system and establish a full-text index for all fields of the directory tree node meta-information, and the external index system is independent of the object storage file system.

13. An electronic device, characterized in that, It includes: One or more processors; A storage device, configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-10.

14. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1-10.

Images