Query method and apparatus, and device

Abstract

A query method and apparatus, and a device. In the present application, a query request is received, wherein the query request is used for obtaining a query target from a file system, a query statement comprises data in the query target or a path of the query target, and the query statement comprises a wildcard character. A prefix compression index is queried on the basis of the query statement to determine an identifier of the query target, wherein the prefix compression index records an identifier of a file or an identifier of a directory, and the file or the directory satisfies any one of the following conditions: the path of the file or the directory comprises a directory file combination, and the file or directory comprises a character combination. On the basis of the query statement, the identifier of the query target is obtained from the prefix compression index, and the query target is obtained by using the identifier of the query target and then fed back to a user. The prefix compression index no longer corresponds to a single word but to combinatorial data, and the query statement can be directly used to locate a specific prefix compression index to obtain the identifier of the query target, thereby reducing the possibility of introducing noise in the query process, and improving the query accuracy.

Description

Query method, device and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on January 16, 2024, with application number 202410065191.3 and application name "A Query Method, Device and Equipment", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of communication technology, and in particular to a query method, apparatus, and device. Background Art

[0004] As the storage capacity of computing devices increases, the amount of data stored in computing devices increases significantly, and the amount of metadata that needs to be managed also increases accordingly. The increase in metadata also makes data querying more difficult.

[0005] Currently, the most common way to query data is to use a full-text search method. Full-text search usually constructs an inverted list based on words, and each word corresponds to an inverted list. The inverted list of any word records one or more files containing the word. When querying a file, the user can provide the sentence that the file needs to contain to indicate the query for files containing the sentence. In order to query the file, the sentence can be segmented to obtain one or more words, and then the inverted list corresponding to each word is found. The inverted lists corresponding to each word are intersected to obtain the file containing the one or more words. The file is the target to be queried. Although this method can speed up the efficiency of file query to a certain extent, its query results may contain a lot of noise and have low accuracy. Summary of the Invention

[0006] The embodiments of the present application provide a query method, apparatus, and device for improving query accuracy.

[0007] In a first aspect, embodiments of the present application further provide a query method that can be executed by a query device. The query device can receive a query request, which can be triggered by a user. The query request is used to obtain a query target from a file system. The query target can be a target file or a target directory. The query request carries a query statement, which includes data in the query target or the path to the query target, and the query statement can include wildcards.

[0008] After receiving the query request, the query device queries the prefix compression index based on the query statement to determine the identifier of the query target. The prefix compression index is an index provided in an embodiment of the present application. For each prefix compression index, it corresponds to a directory file combination or a character combination. The directory file combination includes at least one file name and at least one directory name, or includes at least two directory names, and the character combination includes at least two characters.

[0009] The prefix compression index records the identifier of the file or directory, and the file or directory meets any of the following conditions: the file path contains a directory-file combination, the directory path contains a directory-file combination, the file contains a character combination, and the directory contains a character combination.

[0010] The query statement can be processed to form a directory file combination or a character combination. Based on the query statement, the query device can obtain the identifier of the query target from the prefix compression index. That is, the identifier of the file recorded in the prefix compression index includes the identifier of the target file, or the identifier of the directory recorded in the prefix compression index includes the identifier of the target directory. The query target identifier is then used to obtain and feedback the query target to the user.

[0011] Through the above method, the index queried by the query device no longer corresponds to a single word, but a combination of data. When the query device uses the prefix compression index to determine the query target, it directly uses the query statement to locate one or more specific prefix compression indexes, and then obtains the identifier of the query target. There is no need to perform the operation of intersecting multiple inverted lists, which reduces the possibility of introducing noise during the query process and improves the accuracy of the query.

[0012] In one possible implementation, the query device can also obtain first configuration information, which indicates the types of information included in the metadata. The types of information included in the metadata include some or all of the following: file identification, file path, directory identification, and directory path; the query device can store the metadata of the file or directory based on the first configuration information.

[0013] Through the above method, the query device can adjust the types of information included in the metadata according to the instructions of the first configuration information, ensuring that certain specific types of metadata required can be stored, while other types of metadata not involved in the first configuration information can be not stored, which can effectively utilize storage space and increase the utilization rate of storage resources.

[0014] In one possible implementation, the query device may further obtain second configuration information indicating that prefix compression indexing is enabled. Upon determining that prefix compression indexing needs to be enabled based on the second configuration information, the query device may construct a prefix compression index based on the second configuration information and metadata of the stored file or directory.

[0015] Through the above method, the query device constructs the prefix compression index only when it is determined that the prefix compression index needs to be started. The index construction is more flexible and can be constructed or not constructed according to actual needs.

[0016] In a possible implementation, the second configuration information may further indicate the maximum number of levels of the prefix compression index, which describes the total number of directory names and file names in the directory-file combination, or the total number of characters in the character combination.

[0017] Through the above method, since a prefix compression index corresponds to a directory file combination or character combination, the number of directory file combinations or character combinations determines the number of prefix compression indexes. The maximum number of levels of prefix compression indexes can effectively constrain the number of constructed prefix compression indexes, thereby avoiding the prefix compression indexes occupying more storage space.

[0018] In one possible implementation, the prefix compression index includes at least one index item.

[0019] For any index item, when the index item includes the identifier of a file, the index item also includes the position of the file name in the file path. Optionally, the index item also includes the position of other file names or directory names in the directory file combination in the file path.

[0020] When an index entry includes a directory identifier, the index entry also includes the location of the directory name in the directory path. Optionally, it also includes the location of other file names or directory names in the directory file combination in the directory path.

[0021] Through the above method, the structure of the prefix compression index is relatively simple, which makes it easier for the query device to accurately determine the identifier of the query target.

[0022] In one possible implementation, the prefix compression index includes at least one index item.

[0023] For any index entry, when the index entry includes an identifier of a file, the index entry also includes the position of one or more characters in the character combination in the file.

[0024] Through the above method, the prefix compression index has a relatively simple structure, which makes it easier for the query device to locate the identifier of the query target.

[0025] In one possible implementation, in a scenario where a file's path is used to query a file or a directory's path is used to query a directory's path, when the query device queries the prefix compression index to determine the identifier of the query target, the query device can determine the target prefix compression index from the prefix compression index based on the query statement, and the query statement includes a target directory file combination corresponding to the target prefix compression index.

[0026] Afterwards, the query device determines a first target index item from the target prefix compression index according to the query statement.

[0027] If the first target index includes the identifier of the target file, the identifier of the query target is the identifier of the target file, and for the file name or directory name included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following:

[0028] The first position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target file.

[0029] The second position information included in the first target index item is consistent with the position of the directory name in the query statement, and the second position information is the position of the directory name in the path of the target file.

[0030] If the first target index includes the identifier of the target directory, the identifier of the query target is the identifier of the target directory, and for the file names or directory names included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following:

[0031] The third position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target directory.

[0032] The fourth position information included in the first target index item is consistent with the position of the directory name in the query statement, and the fourth position information is the position of the directory name in the path of the target directory.

[0033] Through the above method, the target prefix compression index can be accurately determined using the query statement, and the position of the file name and / or directory name in the query statement can be accurately determined. During the entire query process, there is no need to intersect between the prefix compression indexes, which reduces the noise that may exist in the query process and ensures the accuracy of the query.

[0034] In one possible implementation, in a scenario where data in a file is used to query a file, when the query device queries the prefix compression index and determines the identifier of the query target, the query device can determine the target prefix compression index from the prefix compression index based on the query statement, and the query statement includes the target character combination corresponding to the target prefix compression index.

[0035] Afterwards, the query device determines a first target index item from the target prefix compression index according to the query statement.

[0036] The first target index includes the identifier of the target file. The identifier of the query target is the identifier of the target file. For the characters included in the target character combination, the first target index item and the query statement satisfy:

[0037] The position information included in the first target index item is consistent with the position of the character in the query statement, and the position information is the position of the character in the target file.

[0038] Through the above method, the target prefix compression index can be accurately determined using the query statement, and the first target index item can be accurately determined using the position of the characters in the query statement. During the entire query process, there is no need to intersect between the prefix compression indexes, which reduces the noise that may exist in the query process and ensures the accuracy of the query.

[0039] In a possible implementation, in addition to using the prefix compression index to determine the identifier of the query target, the query device can also use the inverted list to determine the query target. The specific process is as follows:

[0040] The query device may query the postings list based on the query statement to determine the postings list corresponding to each word in the query statement.

[0041] The query device determines candidate targets included in the posting lists corresponding to the respective words based on the posting lists corresponding to the respective words.

[0042] The query device determines the query target from the candidate targets according to the position of each word in the query sentence and the position of the word recorded in the inverted list corresponding to each word in the candidate targets.

[0043] The query device feeds back the query target to the user.

[0044] Through the above method, when the query device uses the postings list to determine the query target, in addition to intersecting the postings lists, it also uses the position of each word in the query statement to determine the query target from the candidate targets. That is, after intersecting the postings lists, another screening step is performed to ensure that the query target can be accurately located in the end, so that the correct query target can be fed back to the user.

[0045] In one possible implementation, for a word in a query statement, the query target satisfies:

[0046] The position of the word in the query statement is the same as the word recorded in the query target in the inverted list corresponding to the word.

[0047] Through the above method, the position of the word in the query target is consistent with that recorded in the inverted list, ensuring that the determined query target is the correct file or directory that the user needs to query.

[0048] In one possible implementation, the search device searches the prefix compression index based on the query statement of the query target. Before determining the identifier of the query target, further screening may be performed to ensure that the query statement satisfies some or all of the following conditions:

[0049] 1. The frequency of occurrence of a word in the query statement is not greater than the maximum frequency of occurrence of the word.

[0050] 2. The length of the query statement must not exceed the maximum length of the data in the file or the maximum length of the path.

[0051] 3. The position of the word in the query statement is consistent with the preset position of the word.

[0052] By using the above method, filtering is performed before querying the prefix compression index, which can reject some unprocessable query requests in advance and ensure query efficiency.

[0053] In a second aspect, the present application also provides a query method, which can be executed by a query device. The beneficial effects can be found in the relevant description of the first aspect, which will not be repeated here.

[0054] The query device may query a postings list based on a query statement to determine the postings list corresponding to each word in the query statement. Based on the postings list corresponding to each word, the query device determines candidate targets included in the postings list corresponding to each word. Based on the position of each word in the query statement and the position of the word recorded in the postings list corresponding to each word in the candidate targets, the query device determines the query target from the candidate targets. The query device provides feedback of the query target to the user.

[0055] In one possible implementation, for a word in a query statement, the query target satisfies:

[0056] The position of the word in the query statement is the same as the word recorded in the query target in the inverted list corresponding to the word.

[0057] In one possible implementation, a query device receives third configuration information indicating that prefix compression indexing is not enabled. The query device constructs an inverted list based on metadata of a file or directory. If it is determined that prefix compression indexing is not enabled, the inverted list is constructed to ensure that the inverted list can be used for subsequent file or directory queries.

[0058] On the third aspect, the embodiment of the present application further provides a query device, which has the function of implementing the behavior in the method example of the first aspect or the second aspect above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware, or the corresponding software can be executed by hardware. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the structure of the device includes a receiving module, an indexing module and a feedback module, and optionally, a data processing module. These modules can perform the corresponding functions in the method example of the first aspect above. Please refer to the detailed description in the method example for details, which will not be repeated here.

[0059] In a fourth aspect, the present application further provides a computing device comprising a processor and a memory, and may further comprise a communication interface, wherein the processor executes program instructions in the memory to perform the method provided in the first aspect or any possible implementation of the first aspect. Alternatively, the processor executes program instructions in the memory to perform the method provided in the second aspect or any possible implementation of the second aspect. The memory is coupled to the processor and stores computer program instructions and data necessary for determining anomaly detection.

[0060] The communication interface is used to communicate with other devices, such as obtaining query requests, first configuration information, second configuration information, third configuration information, feedback query target detection, etc.

[0061] In a fifth aspect, the present application further provides a computing device comprising an acceleration device, a processor, and optionally, a memory and a communication interface. The processor and the acceleration device cooperate to execute the method provided in the first aspect or any possible implementation of the first aspect. Alternatively, the acceleration device executes the method provided in the first aspect or any possible implementation of the first aspect. The memory is coupled to the processor and stores computer program instructions and data required for determining the query.

[0062] The communication interface is used to communicate with other devices, such as obtaining query requests, first configuration information, second configuration information, third configuration information, feedback query targets, etc.

[0063] or;

[0064] The processor and the acceleration device are configured to execute the method provided by the second aspect or any possible implementation of the second aspect. Or the acceleration device executes the method provided by the second aspect or any possible implementation of the second aspect.

[0065] Communication interface, used to communicate with other devices, such as obtaining query requests, third-party configuration information, and feedback query targets.

[0066] In a sixth aspect, the present application provides a computing device system comprising at least one computing device. Each computing device comprises a memory and a processor. The processor of at least one computing device is configured to access code in the memory to execute the method provided in the first aspect or any possible implementation of the first aspect, or the processor of at least one computing device is configured to access code in the memory to execute the method provided in the second aspect or any possible implementation of the second aspect.

[0067] In a seventh aspect, the present application provides a computer-readable storage medium. When the computer-readable storage medium is executed by a computing device, the computing device executes the method provided in the aforementioned first aspect or any possible implementation of the first aspect, or executes the method provided in the aforementioned second aspect or any possible implementation of the second aspect. The storage medium stores computer program instructions. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, a hard disk drive (HDD), and a solid state drive (SSD).

[0068] In an eighth aspect, the present application provides a computing device program product, the computing device program product including computer program instructions, which, when executed by a computing device, causes the computing device to perform the method provided in the aforementioned first aspect or any possible implementation of the first aspect, or to perform the method provided in the aforementioned second aspect or any possible implementation of the second aspect. The computer program product may be a software installation package, and when it is necessary to use the method provided in the aforementioned first aspect or any possible implementation of the first aspect, or to use the method provided in the aforementioned second aspect or any possible implementation of the second aspect, the computer program product may be downloaded and executed on a computing device.

[0069] In the ninth aspect, the present application also provides a computer chip, which is connected to a memory, and the chip is used to read and execute computer program instructions stored in the memory, execute the methods in the above-mentioned first aspect and each possible implementation of the first aspect, or execute the above-mentioned second aspect and each possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] FIG1 is a schematic diagram of the structure of a query system provided by this application;

[0071] FIG2 is a flow chart of a query method provided by the present application;

[0072] Figures 3A to 3D are schematic diagrams of an information configuration interface provided by this application;

[0073] FIG4 is a schematic diagram of a computing device provided by the present application;

[0074] FIG5 is a schematic diagram of the structure of a query device provided by the present application;

[0075] 6 and 7 are schematic diagrams of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] Before describing a query method, apparatus, and device provided in the embodiments of the present application, some concepts involved in the embodiments of the present application are first clarified.

[0077] (1) File system.

[0078] A file system is a structured data storage and organization method. It uses the concept of "files" to organize data on computing devices. Data used for the same purpose is organized into different file types according to the structure required by different applications. Different suffixes are typically used to denote different file types, and each file is given a memorable name, also known as the "file name." When there are a large number of files, these files are grouped according to a specific method, with each group of files placed in the same directory (or folder). Furthermore, within a directory, there can be subdirectories (called subdirectories or subfolders) in addition to files, forming a tree-like structure. This tree structure has a special name: file system. There are many types of file systems, including FAT / FAT32 / NTFS in Windows and EXT2 / EXT3 / EXT4 / XFS / BtrFS in Linux. In order to facilitate searching, start from the root node and go down the directories one level at a time until you reach the file itself. Use special characters (such as "\" for Windows / DOS and " / " for Unix-like systems) to splice these directory names, subdirectory names, and file names together. This string of characters is called a file path, such as " / etc / systemd / system.conf" in Linux or "C:\Windows\System32\taskmgr.exe" in Windows. The file path is a unique identifier for accessing a specific file. For example, D:\data\file.exe under Windows is the path of a file, which represents the file.exe file in the data directory under the D partition. Similar to files, directories can be understood as a special kind of "file", and directories also have directory paths. The directory path is the information for locating the directory. In an embodiment of the present application, the special characters connecting the directory name, subdirectory name, and file name in the directory path or file path are called separators. The embodiment of the present application does not limit the specific form of the separator.

[0079] (2) Inverted list.

[0080] The inverted list is also called the inverted index or the reverse index. The inverted list can be used in scenarios where records are searched using the values ​​of attributes. Among them, a record can be understood as a carrier of one or more attributes. The meanings represented by the record and the attribute can be different depending on the scenario of searching for a record. For example, in the scenario of searching for a file using the data in the file, the record can be the file, and the attribute can be the data in the file (such as one or more words, one or more sentences in the file). For another example, in the scenario of searching for a directory using a directory path or searching for a file using a file path, the record can be understood as a directory path or a file path, and the attribute can be the directory name, subdirectory name, or file name in the directory path or file path.

[0081] In the postings list, the value of the attribute is used as a keyword to create an index, and the postings list can indicate records containing the value of the attribute. The embodiment of the present application does not limit the specific presentation form of the postings list.

[0082] Here are two ways to present the inverted list:

[0083] 1) Bitmap.

[0084] The bitmap includes multiple bits, each bit corresponds to a record (for example, in a file system scenario, a record can be understood as a file or directory), and the value of each bitmap represents whether the corresponding record contains the value of the attribute.

[0085] For example, in a scenario where data within a file is used to query a file, a bitmap can be created for the data in some files, with different data corresponding to different bitmaps. The bitmap corresponding to any data contains multiple bits, and the number of bits can be the same as the number of files in the file system. Each bit corresponds to a file, and the value of the bit indicates whether the corresponding file contains the data. If the bit value is 1, it indicates that the corresponding file contains the data. If the bit value is 0, it indicates that the corresponding file does not contain the data.

[0086] For another example, in a scenario where a directory path is used to search for a directory or a file path is used to search for a file, a bitmap can be created for some directory names, subdirectory names, or file names, and different directory names (subdirectory names, or file names) can correspond to different bitmaps. The bitmap corresponding to any directory name contains multiple bits, and the number of bits can be the same as the total number of files and directories in the file system. Each bit corresponds to a file or directory, and the value of the bit represents whether the path of the corresponding file or the path of the directory contains the directory name. If the value of the bit is 1, it represents that the path of the corresponding file or the path of the directory contains the directory name. If the value of the bit is 0, it represents that the path of the corresponding file or the path of the directory does not contain the directory name.

[0087] 2) Jump table.

[0088] The skip list includes information about the "records" that contain the attribute's value. That is, the skip list only carries information about records that contain the attribute's value, and does not carry information about records that do not contain the attribute's value. The skip list created for the attribute's value includes multiple index entries, the number of which is the same as the number of records that contain the attribute's value. Each index entry may include identification information for the record, and optionally, each index entry may also include the attribute's position within the record.

[0089] For example, in a scenario where data in a file is used to search for a file, a jump table can be created for the data in some files, and different data can correspond to different jump tables. The jump table corresponding to any data records the file identifier of the file, and optionally, also records the position of the data in the file. For another example, in a scenario where a directory path is used to search for a directory or a file path is used to search for a file, a jump table can be created for some directory names, subdirectory names, or file names, and different directory names (subdirectory names, or file names) can correspond to different jump tables. The jump table corresponding to any directory name records the directory identifier, and optionally, also records the position of the directory name in one or more directory paths. Or the jump table records the file identifier, and optionally, also records the position of the directory name in one or more file paths. Among them, the directory identifier and the file identifier are information for identifying the directory and the file, and are the information required to obtain the directory and the file. In actual applications, the directory identifier and the file identifier can also be replaced with other information required to obtain the directory and the file.

[0090] The structure of the skip list is described below using an inverted index suitable for searching a directory using a directory path or searching a file using a file path as an example.

[0091] Assume that the file system contains four files, and the file paths of the four files are:

[0092] File 1: / abc / bcd / def / efg.

[0093] File 2: / abc / def / efg / bcd.

[0094] File 3: / def / efg / abc / bcd.

[0095] File 4: / def / efg / abc / bcd.

[0096] Then, there may be four inverted lists, which are:

[0097] Jump table 1 (corresponding directory name or file name is abc): 1 / 1, 2 / 1, 3 / 3, 4 / 2.

[0098] Jump table 2 (the corresponding directory name or file name is bcd): 1 / 2, 2 / 4, 3 / 4, 4 / 3.

[0099] Jump table 3 (the corresponding directory name or file name is def): 1 / 3, 2 / 2, 3 / 1, 4 / 1.

[0100] Jump table 4 (the corresponding directory name or file name is efg): 1 / 4, 2 / 3, 3 / 2, 4 / 4.

[0101] In each index entry in the jump list, a " / " is used to separate the file identifier and the directory name or file name corresponding to the jump list in the file path. The file name is before the " / ", and the directory name or file name corresponding to the jump list in the file path is after the " / ".

[0102] It should be noted that the specific presentation method of the index items in the jump table in the above example is only an example. The embodiment of the present application does not limit the presentation method of the index items. Any method that can record the file identifier and the location of the directory name or file name corresponding to the jump table in the file path is applicable to the embodiment of the present application.

[0103] (3) Prefix compression index and the number of levels of prefix compression index.

[0104] An embodiment of the present application provides an index constructed based on a path (such as a file path and a directory path). For the sake of convenience, this index is called a prefix compression index.

[0105] Prefix compression indexes can be used to search for records using multiple attribute values. For a description of attributes and records, see the previous section and will not be repeated here.

[0106] Different prefix compression indexes can be set for different attribute combinations, which include values ​​of multiple attributes. The prefix compression index of an attribute combination indicates the records containing the attribute combination. The embodiment of the present application does not limit the specific presentation form of the prefix compression index.

[0107] Similar to the inverted list, the prefix compression index can also be presented in the form of a bitmap or skip list:

[0108] 1) Bitmap.

[0109] The bitmap includes multiple bits, each bit corresponds to a record (for example, in a file system scenario, a record can be understood as a file or directory), and the value of each bitmap represents whether the corresponding record contains the attribute combination.

[0110] For example, in a scenario where the data in a file is used to query a file, a bitmap can be created for some characters in the file (the characters here are used to refer to certain specific data included in the file or single characters formed after word segmentation of the data in the file, such as words, characters, symbols, etc. included in the file). Different character combinations can correspond to different bitmaps, where a character combination contains multiple characters. The bitmap corresponding to any character combination contains multiple bits, and the number of bits can be the same as the number of files in the file system. Each bit corresponds to a file, and the value of the bit represents whether the corresponding file contains the character combination. If the value of the bit is 1, it represents that the corresponding file contains the character combination. If the value of the bit is 0, it represents that the corresponding file does not contain the character combination.

[0111] It should be noted that the embodiments of the present application do not limit the specific form of any character in the character combination. For example, the character can be a letter, an English word, a Chinese character, a Chinese vocabulary, a punctuation mark, a Greek symbol, etc.

[0112] For another example, in a scenario where a directory path is used to search for a directory or a file path is used to search for a file, a bitmap can be created for some directory names, subdirectory names, or file names. Different directory-file combinations can correspond to different bitmaps, where a directory-file combination can include multiple directory names or at least one directory name and at least one file name.

[0113] A bitmap corresponding to any directory-file combination. This bitmap contains multiple bits, which can be the same as the total number of files and directories in the file system. Each bit corresponds to a file or directory, and the value of the bit indicates whether the path to the corresponding file or directory contains the directory name. If the bit value is 1, the path to the corresponding file or directory contains the directory name. If the bit value is 0, the path to the corresponding file or directory does not contain the directory name.

[0114] 2) Jump table.

[0115] The skip list includes relevant information about "records" containing this attribute combination. That is, the skip list only carries relevant information about "records" containing this attribute combination, and does not carry relevant information about "records" that do not contain this attribute combination. The skip list established for this attribute combination includes multiple index items, and the number of index items is the same as the number of "records" containing this attribute combination. For example, each index item can include identification information for the record, and optionally, each index item can also include the location of some or all attributes in the attribute combination in the record.

[0116] For example, in a scenario where data in a file is used to query a file, a jump table can be established for character combinations in some files, and different character combinations can correspond to different jump tables. The jump table corresponding to any character combination records the file identifier of the file, and optionally, also records the position of one or more characters in the character combination in the file. For another example, in a scenario where a directory path is used to search for a directory or a file path is used to search for a file, a jump table can be established for a directory-file combination, and different directory-file combinations can correspond to different jump tables. The jump table corresponding to any directory-file combination records the directory identifier and the position of part or all of the directory-file combination in one or more directory paths, or the jump table records the file identifier and or the position of part or all of the directory-file combination in one or more file paths. Among them, the directory identifier and the file identifier are information for identifying a directory and a file, and are the information required to obtain the directory and the file. In actual applications, the directory identifier and the file identifier can also be replaced with other information required to obtain the directory and the file.

[0117] Note: Subdirectory is a special kind of "directory" and is a relative concept used when describing the structure of a directory. A subdirectory refers to the next level "directory" included in a directory.

[0118] The level of the prefix compression index describes the number of corresponding attributes. The embodiment of the present application does not limit the value of the level of the prefix compression index. For example, the level of the prefix compression index is equal to the number of attributes recorded by the prefix compression index. For another example, the level of the prefix compression index is equal to the number of attributes recorded by the prefix compression index minus one. When the level of the prefix compression index is equal to 0, the prefix compression index is the inverted list mentioned above.

[0119] The following describes the structure and levels of a prefix compression index in the form of a skip list, taking a prefix compression index suitable for searching a directory using a directory path or searching a file using a file path as an example.

[0120] Let’s take the four files mentioned above as an example.

[0121] Assume that the number of levels of the prefix compression index is equal to the number of attributes recorded by the prefix compression index minus one. Then, the maximum number of levels of the prefix compression index is three. The " / * / " in the directory name or file name corresponding to the prefix compression index is used to represent any character. The structure of the index item in the prefix compression index is similar to the structure of the index item in the aforementioned inverted table. The following is an example in which the index item only records the position of the last directory name in the directory file combination corresponding to the prefix compression index in the file path. Similarly, the embodiment of the present application does not limit the specific structure of the index item in the prefix compression index.

[0122] There are multiple first-level prefix compression indexes, two of which are listed below:

[0123] Level 1 prefix compression index 1 (the corresponding directory file combination is abc / * / bcd): 1 / 2, 2 / 4.

[0124] Level 1 prefix compression index 2 (corresponding directory file combination is def / * / bcd): 3 / 4, 4 / 3.

[0125] There are multiple secondary prefix compression indexes, one of which is listed below:

[0126] Secondary prefix compression index 1 (corresponding directory file combination is abc / bcd / * / efg): 1 / 4, 2 / 3.

[0127] There is a three-level prefix compression index, one of which is listed below:

[0128] Level 3 prefix compression index 1 (corresponding directory file combination is abc / bcd / def / efg): 1 / 4.

[0129] As can be seen from the foregoing description, in practice, without considering the storage space occupied by prefix compressed indexes, the number of compressed indexes of different levels is related to the number of directory file combinations that can be constructed. If the storage space occupied by prefix compressed indexes is considered, the number of compressed indexes of different levels can be reduced as needed. For example, if the directory file combinations corresponding to the compressed indexes must include the root directory, then there is no need to construct prefix compressed indexes corresponding to directory file combinations that do not include the root directory.

[0130] (4) Metadata.

[0131] Metadata, also known as intermediary data or relay data, is data about data, mainly information describing data properties, such as the data address, data modification history, data size, data creation date, etc.

[0132] Here, the storage and organization of data in the form of a file system is taken as an example. The metadata involved in the file system includes file metadata, directory metadata, etc.

[0133] File metadata describes the attributes of the data within the file, such as the file name, file path (also known as the file address), data modification history, file size, file identifier, etc. Directory metadata describes directory attribute information, such as the directory name, directory modification history, directory path, directory identification (ID), directory creation time, directory modification time, access time, and group to which the directory belongs.

[0134] In an embodiment of the present application, the user is allowed to define the types of information included in the metadata, and the query device can organize and store the metadata according to the information types defined by the user.

[0135] (5) Participle.

[0136] When performing a file or directory search, it is usually necessary to first perform word segmentation on the information related to the query target carried in the query request (in this embodiment of the application, the information related to the query target is referred to as a query statement). In other words, the query statement carried in the query request is divided into word units and the query statement is converted into one or more words. The case where the query statement is converted into a single word is a special case of "word segmentation". This is usually when the query request contains a small number of words related to the query target (for example, only one character) and cannot be further divided.

[0137] The query statement includes but is not limited to: the data in the query target and the path to the query target.

[0138] Typically, the path of a query target includes delimiters, and each directory or file name separated by the delimiter can be understood as a word segmentation. The data in the query target carried in the query request (which may include wildcards) needs to be word segmented. For example, if the query carried in the query request is "apple tree", the word "apple tree" can be segmented into "apple", "tree", "apple", "fruit", etc.

[0139] It should be noted that in the embodiments of this application, no special distinction is made between words and characters formed after word segmentation. Words formed after word segmentation can be understood as a type of character, and a specific representation of a character can be represented as a word. In the embodiments of this application, words and characters have the same meaning. Generally, the term "word" is used in descriptions involving word segmentation or inverted list query processes for data, and the term "character" is used in descriptions of the aforementioned compressed index design.

[0140] As shown in FIG1 , it is a schematic diagram of the structure of a query system provided by an embodiment of the present application. The query system includes a query device 100 and a storage device 200 .

[0141] The query device 100 can interact with the user, store the metadata of the data (such as the metadata of the file or the metadata of the directory) in the storage device 200 according to the user's instructions, and can also process the query request triggered by the user, and query the file or directory from the storage device 200 based on the pre-built inverted list and / or prefix compression index.

[0142] Function 1: The query device 100 has a data processing function.

[0143] The data processing function of the query device 100 is mainly manifested in the processing of metadata and the construction of indexes.

[0144] 1) Metadata processing.

[0145] Query device 100 is user-friendly and allows users to configure the types of information included in metadata based on their needs. After obtaining the information types included in the user-configured metadata, query device 100 processes the metadata. The processed metadata includes information types consistent with the user-configured metadata. Query device 100 can store the processed metadata in storage device 200.

[0146] 2) Index construction.

[0147] After processing the metadata, the query device 100 can also construct an index based on the metadata. In the embodiment of the present application, the query device 100 can construct two indexes, one index is an inverted index, and the other index is a prefix compression index.

[0148] From the above description of inverted index and prefix compressed index, we can see that inverted index and prefix compressed index are applicable to different query scenarios. In different query scenarios, the inverted index includes different content, and the prefix compressed index also includes different content in different query scenarios.

[0149] When constructing an index, the query device 100 can construct an inverted index and a prefix compression index suitable for searching for files using data in the file, and can also construct an inverted index and a prefix compression index suitable for searching for directories using directory paths or searching for files using file paths.

[0150] The characters in the file corresponding to the inverted index and prefix compression index used for searching files using data in the file may be different or not completely the same. For example, the inverted index corresponds to only one character in the file, while the prefix compression index corresponds to a character combination, and the character combination includes at least two characters.

[0151] The file names or directory names corresponding to the inverted index and prefix compression index applicable to scenarios where a directory is searched using a directory path or a file is searched using a file path may be different or not completely the same. For example, the inverted index corresponds to only one directory name or one file name in the file system, while the prefix compression index corresponds to a directory-filename combination, where the directory-filename combination includes at least two directory names or at least one directory name and at least one file name in the file system.

[0152] When constructing an index, the query device 100 can construct an inverted index suitable for searching for files using data in the file, and a prefix compression index suitable for searching for a directory using a directory path or searching for a file using a file path.

[0153] Furthermore, when constructing a prefix compression index, the query device 100 may construct a prefix compression index in the form of a bitmap or a skip list, or in the form of a bitmap or a skip list. Similarly, when constructing an inverted list, the query device 100 may construct an inverted list in the form of a bitmap or a skip list, or in the form of a bitmap or a skip list.

[0154] Function 2: The query device 100 has a data query function.

[0155] The query device 100 is capable of processing a query request triggered by a user, wherein the query request is used to query a target directory or target file. For ease of explanation, the target directory or target file requested by the query request is referred to as the query target. The query request may include data in the query target or the path to the query target.

[0156] It should be noted that the query device 100 in the embodiment of the present application supports fuzzy queries, that is, the query request can carry data in the query target or the path of the query target can contain wildcards, wherein the wildcard can replace any character. The embodiment of the present application does not limit the specific form of the wildcard. For example, the wildcard can be "?", "*", space, etc. In other words, the data in the query target that can be carried in the query request can be incomplete data, and the path of the query target carried in the query request can be an inaccurate path. Therefore, in the embodiment of the present application, the query device 100 can use incomplete data and inaccurate paths to find the query target. If the query request can carry an accurate path, the query device 100 can directly find the query target based on the accurate path. If the query request can carry complete data in the file (that is, data that does not include wildcards), the query device 100 can not perform word segmentation on the data, but directly query the index based on the complete data to determine the identifier of the query target and then find the query target.

[0157] After receiving a query request, the query device 100 can search for the query target based on the constructed index. When the query request contains data from the query target, the query device 100 can search an inverted index or a prefix-compressed index based on the data in the query target to determine the path to the query target and then use the path to retrieve the query target. The inverted index or prefix-compressed index is an index constructed by the query device 100 that is suitable for searching files using data in the files.

[0158] When the query request carries the path of the query target (e.g., a path containing wildcards), the query device 100 can query the inverted index or prefix compression index based on the path of the query target to determine the identifier of the query target and obtain the query target. The inverted index or prefix compression index is an index constructed by the query device 100 and is suitable for searching files using file paths or searching directories using directory paths.

[0159] In addition, since the query device 100 can construct an inverted index or a prefix compression index in the form of a bitmap and a skip list, the embodiment of the present application takes the construction of an inverted list in the form of a bitmap and a skip list as an example for explanation.

[0160] When the postings list is an index suitable for searching for files using data in the file, the query device 100 queries the postings list based on the data in the query target. The query device 100 can simultaneously query the bitmap and skip list; or it can query only the bitmap or skip list. For example, when the shortest length of the postings list corresponding to the data in the query target (that is, the shortest length of the postings list corresponding to the words in the query target after word segmentation) is greater than a first threshold, the query device 100 queries the bitmap. When the shortest length of the postings list corresponding to the data in the query target is not greater than the first threshold, the query device 100 queries the skip list. The length of the postings list describes the total number of file identifiers and directory identifiers recorded in the postings list.

[0161] When the posting list is an index suitable for searching files using a file path or querying a directory using a directory path, the query device 100 queries the posting list based on the path of the query target. The query device 100 can synchronously query the bitmap and the skip list; or it can only query the bitmap or the skip list. For example, when the shortest length of the corresponding posting list in the path of the query target is greater than the second threshold, the query device 100 queries the bitmap; when the shortest length of the corresponding posting list in the path of the query target is not greater than the second threshold, the query device 100 queries the skip list.

[0162] Before performing a data query, the query device 100 may also perform query pre-filtering. The so-called "query pre-filtering" refers to determining whether the query request can be processed or whether it is possible to find the query target before performing a data query.

[0163] In the embodiment of the present application, the query device 100 may perform "query pre-filtering" from some or all of the following aspects:

[0164] Aspect 1: word frequency.

[0165] For any word, there is always a maximum frequency of occurrence of the word in the file and a maximum frequency of occurrence of the word in the file path (or directory path). For example, the maximum number of occurrences of "apple tree" in the file is 10, that is, in any of the multiple files in the file system, "apple tree" appears at most 10 times. Another example, the maximum number of occurrences of "New File 1" in the file path (or directory path) is 2, that is, in the path of any file (or directory) in the file system, "New File 1" appears at most twice.

[0166] The query device 100 can filter the query request by word frequency. If it is determined that any word in the data of the query target carried in the query request or the path of the query target is greater than the maximum frequency of occurrence of the word, the query request is rejected and the query request is no longer processed. Among them, the maximum frequency of occurrence of the word is the maximum number of occurrences of the word in a single file in the file system (applicable to the scenario of finding files using the data in the file), and the maximum frequency of occurrence of the word is the maximum number of occurrences of the word segmentation in a single file path or directory path in the file system (applicable to the scenario of finding directories using directory paths or finding files using file paths).

[0167] Aspect two: The length of the data in the file or the length of the directory path (or file path).

[0168] For the data in any file in the file system (such as a statement in a file), there is always a maximum value for the length of the data. For the path of any file in the file system or the path of any directory, there is also always a maximum value for the length of the directory path (or file path).

[0169] If the length of the data or path in the query target carried in the query request exceeds the corresponding maximum value, the query request can be rejected.

[0170] Aspect three: Word position.

[0171] For a certain word in the data in any file in the file system (such as a statement in a file), the position of the word in the data usually appears at a fixed position. For example, the words "le" and "ma" always appear at the end of the data. For the path of any file in the file system or the path of any directory, a certain directory name or file name in the directory path (or file path) usually appears at a fixed position. For example, the root directory always appears at the beginning of the directory path (or file path).

[0172] If the position of a word in the data of the query target carried in the query request is different from the preset position of the word, the query request can be rejected. If the position of a word in the path of the query target carried in the query request is different from the preset position of the word, the query request can be rejected. The preset position of the word is the position of the word in the data of the file in the file system (applicable to the scenario of searching for files using data in the file), and the preset position of the word is the position of the word in the file path or directory path in the file system (applicable to the scenario of searching for directories using directory paths or searching for files using file paths).

[0173] The embodiments of the present application do not limit the specific form of the query device 100. The query device 100 can be a hardware device, a single computing device, or a cluster including multiple computing devices. The query device 100 can also be a hardware component in a computing device, such as a processor (such as a central processing unit (CPU), a data processing unit (DPU)), an offload card, an accelerator card, etc. in the computing device. The query device 100 can be a software device, such as software for managing files, such as file management software, database management software, etc.

[0174] The storage device 200 is used to store data and metadata. The embodiments of the present application do not limit the specific type of the storage device 200. The storage device 200 can store data or metadata under the instruction of the query device 100, and can also transmit data to the query device 100 under the instruction of the query device 100. For example, after the query device 100 finds a file or directory, it can execute the storage device 200 to transmit the file or directory to the query device 100 according to the identifier of the file or the description of the directory. The storage device 200 can be a computing device or a computing device cluster with a storage function, and the storage device 200 can also be a component with a storage function in a computing device, such as a hard disk, a magnetic disk, etc. Any device with a data storage function is applicable to the embodiments of the present application.

[0175] The embodiments of the present application do not limit the deployment methods of the storage device 200 and the query device 100. The storage device 200 and the query device 100 can be deployed in the same computing device; for example, the query device 100 can be a processor (such as a CPU, a DPU) in the computing device, and the storage device 200 is a hard disk in the computing device. The storage device 200 and the query device 100 can be deployed in different computing devices; for example, the storage device 200 and the query device 100 can be deployed in a storage system, the query device 100 is a computing node in the storage system (with data computing functions, undertaking computing tasks in the storage system), and the storage device 200 is a storage node in the storage system (with data storage functions).

[0176] The query method provided in the embodiment of the present application is described below with reference to FIG2 . The query method includes two parts: one part is the metadata storage and index construction process, which can be specifically seen in steps 201 to 204 ; the other part is the data query process, which can be specifically seen in steps 205 to 213 .

[0177] In this section, data query can be performed in two ways. One way is to use a prefix compression index to perform data query, as detailed in steps 207 to 209. The other way is to use an inverted index to perform data query, as detailed in steps 210 to 213. These two ways are applicable to scenarios where data in a file is used to search for a file, as well as scenarios where a directory path is used to search for a directory, or a file path is used to search for a file. In the embodiment shown in FIG2 , an inverted index is used to perform data query in scenarios where data in a file is used to search for a file, and a prefix compression index is used to perform data query in scenarios where a directory path is used to search for a directory, or a file path is used to search for a file.

[0178] Using prefix compression indexes to query data in a file search scenario is similar to using prefix compression indexes to query data in scenarios where data in a file is used to search for a directory or a file using a file path. The only difference is that the prefix compression indexes in these two scenarios correspond to different information: in the former scenario (searching for a file using data in a file), the corresponding information is a character combination, while in the latter scenario (searching for a directory using a directory path or a file using a file path), the corresponding information is a directory-file combination. The principles of data query are similar and will not be repeated here.

[0179] In scenarios where you use a directory path to find a directory or a file path to find a file, using a posting table to query data is similar to using a posting table to query data in scenarios where you use data in a file to find a file. The only difference is that the posting table in these two scenarios corresponds to different information: in the former scenario (searching for a file using data in a file), the corresponding information is data, while in the latter scenario (searching for a directory using a directory path or searching for a file using a file path), the corresponding information is a directory or a file. The principles of data query are similar and will not be repeated here.

[0180] Step 201: The query device 100 obtains first configuration information provided by a user. The first configuration information indicates the type of information included in the metadata.

[0181] Metadata is used to describe data, and the types of information in metadata can cover a wide range of information that can characterize data attributes. In practical applications, the more types of information metadata includes, the more storage space it occupies. To save storage space, metadata can include some relatively important information. In addition, based on the application scenario of the data itself, some information that characterizes data attributes is more important, while some information that characterizes data attributes can be ignored. For example, in a database scenario, information such as data modification records, data addresses, data table names, or file names is more important, while the number of data modifications can often be ignored.

[0182] In the embodiment of the present application, the query device 100 provides a user with an interface for configuring the type of information included in the metadata. Through the interface, the user can transmit the first configuration information to the query device 100.

[0183] It is worth noting that the interface provided by the query device 100 to the user refers to the function provided by the query device 100 to the user. The embodiment of the present application does not limit the specific form of the interface. The interface can be expressed as a dedicated command for providing the first configuration information, and the interface can also be expressed as a visual interface for the user.

[0184] Figure 3A shows a schematic diagram of a configuration interface provided by an embodiment of the present application, in which a user can select or enter the type of information included in the metadata. The user can configure the type of information included in the file metadata and the directory metadata separately.

[0185] Step 202: The query device 100 obtains second configuration information provided by the user. The second configuration information indicates whether to enable prefix compression index and the maximum number of levels of the prefix compression index.

[0186] The existence of prefix compression index can realize efficient data query, but prefix compression index will take up some storage space, and the larger the number of levels of the prefix compression index, the higher the efficiency of data query, and the larger the storage space occupied. In the embodiment of the present application, the user is allowed to choose to enable or disable the prefix compression index according to his or her actual needs. Furthermore, if the user chooses to enable the prefix compression index, the user is also allowed to configure the maximum number of levels of the prefix compression index.

[0187] In the embodiment of the present application, the query device 100 provides a user with an interface for configuring a prefix compression index, through which the user can transmit the second configuration information to the query device 100 .

[0188] Similar to the interface for configuring the information types included in the metadata provided above, the interface for configuring prefix compression indexes merely describes a configuration function provided by the query device 100 for the user. The embodiment of the present application does not limit the specific form of the interface.

[0189] As shown in FIG3B , it is a schematic diagram of a configuration interface provided in an embodiment of the present application, in which the user can choose whether to enable prefix compression index and the maximum number of levels of the prefix compression index.

[0190] The embodiment of the present application does not limit the execution order of step 201 and step 202. Step 201 may be executed first and then step 202, or step 202 may be executed before step 201. Of course, in actual applications, the query device 100 may also execute steps 201 and 202 simultaneously. The query device 100 allows the user to complete the configuration of the metadata information type and the prefix compression index at one time, that is, the user simultaneously provides the first configuration information and the second configuration information.

[0191] As shown in Figure 3C, it is a schematic diagram of a configuration interface provided in an embodiment of the present application. In this configuration interface, the user can not only check the type of information included in the metadata, but also choose whether to enable prefix compression index and the maximum number of levels of the prefix compression index.

[0192] In the above description, the first configuration information and the second configuration information provided by the user are used as an example for explanation. The query device 100 also allows the user to complete the configuration of other information. For example, the query device 100 allows the user to complete the resource configuration of the data query, wherein the resource indication of the data query is the maximum resource that the query device 100 can occupy to implement the data query. The data query resource includes but is not limited to: the number of processors, the number of processor cores, the size of the memory, the cluster setting (used to indicate the cluster or node in the cluster where the query device 100 needs to be deployed), and the backup setting (such as the data backup method).

[0193] In the embodiment of the present application, step 201 or step 202 is an optional step. The query device 100 may pre-store the information types included in the metadata. The query device 100 may also enable prefix compression indexing by default, and store a preset maximum number of prefix compression index levels in the query device 100. The query device 100 may use the pre-stored information types included in the metadata and / or the maximum number of prefix compression index levels to perform subsequent steps.

[0194] Step 203: The query device 100 obtains metadata of the data and saves the metadata of the user in the storage device 200 based on the first configuration information.

[0195] The query device 100 provides a data interface for the user, through which the user can transmit data to the query device 100. After obtaining data through the data interface and generating metadata for the data based on the data, the query device 100 can save the metadata according to the first configuration information, that is, retain the information type indicated by the first configuration information in the metadata and delete the information type indicated by the first configuration information.

[0196] Step 204: The query device 100 constructs an index for the user's metadata according to the second configuration information.

[0197] If the second configuration information indicates that prefix compression indexing is enabled, the query device 100 may construct a prefix compression index for the metadata. If the second configuration information also indicates a maximum number of levels for prefix compression indexing, the number of levels of the prefix compression index constructed by the query device 100 shall not exceed the maximum number of levels for prefix compression indexing indicated by the second configuration information. Optionally, the query device 100 may also construct an inverted index for the user's metadata.

[0198] If the second configuration information indicates that the prefix compression index is not enabled, the query device 100 can construct an inverted index for the user's metadata.

[0199] When the query device 100 executes step 204, the prefix compressed index constructed can be a prefix compressed index suitable for a scenario where data in a file is used to query a file, or a prefix compressed index suitable for a scenario where a directory path is used to search for a directory or a file path is used to search for a file, or a prefix compressed index that includes both a prefix compressed index suitable for a scenario where data in a file is used to query a file and a prefix compressed index suitable for a scenario where a directory path is used to search for a directory or a file path is used to search for a file. In the embodiment shown in FIG2 , the prefix compressed index constructed by the query device 100 is a prefix compressed index suitable for a scenario where a directory path is used to search for a directory or a file path is used to search for a file as an example.

[0200] In addition, the embodiment of the present application does not limit the existence form of the prefix compression index constructed by the query device 100. The query device 100 can construct prefix compression indexes in the form of skip tables and bit tables respectively, or can only construct prefix compression indexes in the form of skip tables or bit tables.

[0201] Similarly, when the query device 100 executes step 204, the inverted index constructed can be an inverted index suitable for a scenario where data in a file is used to query a file, or an inverted index suitable for a scenario where a directory path is used to search for a directory or a file path is used to search for a file, or an inverted index that includes both an inverted index suitable for a scenario where data in a file is used to query a file and an inverted index suitable for a scenario where a directory path is used to search for a directory or a file path is used to search for a file. In the embodiment shown in FIG2 , the inverted index constructed by the query device 100 is an inverted index suitable for a scenario where data in a file is used to query a file as an example.

[0202] In addition, the embodiment of the present application does not limit the form of the posting list constructed by the query device 100. Here, it is assumed that the query device 100 constructs the posting list in the form of a skip list and a bit table.

[0203] At this point, the query device 100 has completed the preparation operations before executing the data query. After building the index, the query device 100 can receive and process the query request.

[0204] Step 205: The query device 100 receives a query request triggered by a user. The query request carries a query statement. The query statement can be data in the query target or a path to the query target.

[0205] The embodiment of the present application does not limit the manner in which the query apparatus 100 performs step 205. For example, the user may send the query request to the query apparatus 100 via a computing device deployed on the user side. For another example, the user may directly interact with the query apparatus 100 and directly trigger the query apparatus 100 to generate a query request.

[0206] As shown in FIG3D , the query device 100 of the present embodiment provides a visual query interface for the user. In this query interface, the user can choose to query a file using data in the file by typing the data in the query target in the corresponding location. The user can also choose to query a target file (or target directory) using the target file path (or target directory path) by typing the path of the query target in the corresponding location.

[0207] The query device 100 supports fuzzy query, that is, when the user types the data in the query target or the path of the query target, the user can enter a wildcard to replace any character.

[0208] Step 206: The query device 100 performs query pre-filtering on the query request. Step 206 is optional. In actual applications, the query device 100 may not perform query pre-filtering and directly execute subsequent steps.

[0209] The query device 100 may perform query pre-filtering on the query request from the following parts or all.

[0210] Filter 1: Filter the query request based on word frequency.

[0211] If the query request carries data in the query target, the data in the query target can be segmented to obtain one or more words. The query device 100 stores the maximum frequency of occurrence of each word. The maximum frequency of occurrence of any word is the maximum number of times the word appears in a single file in the file system.

[0212] For one of the words, the query device 100 determines whether the frequency of occurrence of the word in the data in the query target carried in the query request is greater than the maximum frequency of occurrence of the word segment. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0213] If the query request carries the path of the query target, the delimiter in the path of the query target is separated into individual words. The query device 100 stores the maximum frequency of occurrence of each word. The maximum frequency of occurrence of any word is the maximum number of times the word appears in a single file path or directory path in the file system.

[0214] For one of the words, the query device 100 determines whether the frequency of occurrence of the word in the data in the query target carried in the query request is greater than the maximum frequency of occurrence of the word. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0215] Filter 2: Filter the query request based on the length of the data in the file or the length of the directory path (or file path).

[0216] If the query request carries data in the query target, the query device 100 stores a maximum length of the data, which is the maximum length of the data in a single file in the file system.

[0217] The query device 100 determines whether the data length of the query target carried in the query request is greater than the maximum length of the data. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0218] If the query request carries the path of the query target, the query device 100 stores the maximum length of the path, which is the maximum length of a single file path or directory path in the file system.

[0219] The query device 100 determines whether the path length of the query target carried in the query request is greater than the maximum length of the path. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0220] Filter 3: Filter query requests based on word position.

[0221] If the query request carries data in the query target, the data in the query target can be parse to obtain one or more words. The query device 100 stores preset positions of some or all of the words. The preset position of any word is the position of the word in the data contained in the file in the file system.

[0222] For one of the words, the query device 100 determines whether the position of the word in the data in the query target carried in the query request is different from the preset position of the word segmentation. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0223] If the query request carries the path of the query target, the delimiter in the path of the query target is separated into individual words. The query device 100 stores the preset positions of some or all of the words, and the preset position of any word is the position of the word in the file path or directory path of the file system.

[0224] For any word, the query device 100 determines whether the position of the word in the data in the query target carried in the query request is different from the preset position of the word. If so, the query request is rejected. Otherwise, the query request passes the filtering and can be processed.

[0225] When the query request passes the query pre-filter, subsequent steps can be executed; otherwise, the query device 100 rejects the query request and informs the user that the query target cannot be found.

[0226] By pre-filtering queries, some query requests that cannot be processed can be rejected in advance, avoiding the need to execute subsequent steps for these query requests, which can effectively improve data query efficiency.

[0227] The following describes an example in which a query request carries a path of a query target, and illustrates how the query device 100 processes the query request based on a prefix compression index.

[0228] Step 207: The query device 100 determines the target prefix compression index based on the query path prefix compression index carried in the query request. The query path includes the target directory file combination corresponding to the target prefix compression index. In other words, the query path includes the directory names and file names in the target directory file combination.

[0229] Since the query device 100 has constructed one or more prefix compression indexes in advance, the query device 100 needs to first locate the target prefix compression index that records the query target from the one or more prefix compression indexes.

[0230] Taking the specific example of the prefix compression index described above as an example, assuming that the query target path carried in the query request is / abc / bcd / * / , the query device 100 can query the prefix compression index (existing in the form of a skip list) corresponding to the directory file combination abc and bcd, that is, the first-level prefix compression index 1.

[0231] Step 208: The query device 100 determines a first target index item from the target prefix compression index according to the path of the query target, where the identifier of the query target is recorded.

[0232] If the first target index item includes the identifier of the target file, then some position information included in the first target index item is consistent with the position of the directory name or file name in the query statement.

[0233] Specifically, for a certain file name in the target directory file combination (the file name can be the file name of the target file or the file name of other files except the target file), if the first target index item includes the position of the file name in the path of the target file, then the position of the file name included in the first target index item in the path of the target file is consistent with the position of the file name in the query statement. Taking the query target path as / abc / bcd / * / as an example, if the first target index item includes the position of the file name bcd in the path of the target file, then the position of the file name abc in the path of the target file needs to be the second position, which is consistent with the position of the file name bcd in the query statement.

[0234] For a directory name in the target directory file combination, if the first target index item includes the location of the directory name in the path of the target file, then the location of the directory name included in the first target index item in the path of the target file is consistent with the location of the directory name in the query statement. Taking the query target path as / abc / bcd / * / as an example, if the first target index item includes the location of the directory name abc in the path of the target file, then the location of the directory name abc in the path of the target file must be the first, which is consistent with the location of the directory name abc in the query statement.

[0235] If the first target index item includes an identifier of a target directory, then some position information included in the first target index item is consistent with the position of the directory name or file name in the query statement.

[0236] Specifically, for a certain file name in the target directory file combination, if the first target index item includes the position of the file name in the path of the target directory, then the position of the file name included in the first target index item in the path of the target directory is consistent with the position of the file name in the query statement.

[0237] For a directory name in the target directory file combination (the directory name can be the file name of the target directory or the directory name of a directory other than the target directory), if the first target index item includes the position of the directory name in the path of the target directory, then the position of the directory name included in the first target index item in the path of the target directory is consistent with the position of the target name in the query statement.

[0238] Therefore, the paths of each file or directory recorded in the target prefix compression index determined in step 207 all contain the names of each file or directory in the path of the query target. The query device 100 needs to further analyze the target prefix compression index and determine the query target from the files or directories recorded in the target prefix compression index.

[0239] The query device 100 determines the first target index item based on the position of the file name or directory name in the path of the query target and the position of at least one file name or directory name included in the corresponding directory file combination recorded in the target prefix compression index in the file path.

[0240] Here, the position of a directory name included in the corresponding directory file combination recorded in the target prefix compression index in the file path is taken as an example for explanation. The query device 100 can find an index item from the target prefix index whose position of the recorded directory name in the file path is consistent with the position of the directory name in the path of the query target. This index item is the first target index item.

[0241] Step 209: The query device 100 obtains the query target according to the first target index item, and feeds back the query target to the user.

[0242] After determining the first target index item, the query device 100 may determine the identifier of the query target from the first target index item, and the query device 100 may obtain the query target from the storage device 200 according to the identifier of the query target.

[0243] The following describes an example in which a query request carries data in a query target, and illustrates how the query device 100 processes the query request based on an inverted index.

[0244] Step 210: The query device 100 performs word segmentation on the data in the query target, converting the data in the query target into one or more words. The present embodiment does not limit the manner in which the query device 100 performs word segmentation on the data in the query target. For example, the query device 100 may employ an n-gram algorithm to perform word segmentation on the data in the query target, where n is the number of words obtained after word segmentation on the data in the query target.

[0245] If the data in the query target is converted into only one word, step 212 can be directly executed (the file recorded in the posting list corresponding to the word is the candidate file). If the data in the query target is converted into multiple words, step 211 can be executed.

[0246] Step 211: The query device 100 determines the posting list corresponding to each word, intersects the posting lists corresponding to multiple words, and determines the candidate files recorded in each posting list.

[0247] After obtaining multiple words, the query device 100 can determine the postings list corresponding to each word from the pre-constructed postings list. Intersecting the postings lists corresponding to multiple words refers to finding files that are recorded in all postings lists corresponding to the multiple words. Files that are recorded in all postings lists corresponding to the multiple words are candidate files.

[0248] Here, it is assumed that the query device 100 obtains two word segments, and determines from the pre-constructed inverted tables that the inverted tables corresponding to each word are Table 1 and Table 2, respectively. Table 1 records file 1, file 2, and file 3, and Table 2 records file 2, file 3, and file 4. Then, after intersecting Table 1 and Table 2, the candidate files obtained are file 3 and file 2.

[0249] Since the query device 100 constructs a posting list in the form of a skip list and a bit map, the query device 100 can synchronously perform step 211 on the skip list and the bit map. That is, the query device 100 not only intersects the bit map corresponding to each word to determine the candidate files recorded in each posting list, but also synchronously intersects the skip list corresponding to each word to determine the candidate files recorded in each posting list. If the intersection of the bit map corresponding to each word can determine the candidate files faster, then the bit map corresponding to each word is used to determine the candidate files and then perform the subsequent steps. If the intersection of the skip list corresponding to each word can determine the candidate files faster, then the skip list corresponding to each word is used to determine the candidate files and then perform the subsequent steps.

[0250] The query device 100 may also perform step 211 on only one of the skip list and the bitmap. For example, when the shortest length of the postings list corresponding to a word segmentation is greater than a first threshold, the query device 100 searches the bitmap and performs step 211 on the bitmap. When the shortest length of the postings list corresponding to a word segmentation is not greater than the first threshold, the query device 100 searches the skip list.

[0251] Step 212: The query device 100 determines the target file from the candidate files according to the position order of the words recorded in the posting list in the candidate files and the position of the words in the data of the query target.

[0252] In addition to recording the file identifier, each index entry in the postings list also records the position of the data in the file corresponding to the term in the postings list. For any candidate file, the query device 100 can determine whether the order of the terms in the candidate file and the positions of the data in the query target are consistent. If they are consistent, the candidate file is the target file; otherwise, the candidate file is not the target file.

[0253] Step 213: The query device 100 feeds back the target file to the user.

[0254] It should be noted that, in the embodiment of the present application, the query method provided by the embodiment of the present application is only described with two query scenarios: using a path to query data (such as using the path of a file to query a document or using the path of a directory to query a directory), and using the data in a file to query a file. In fact, the query method provided by the embodiment of the present application is also applicable to other scenarios, such as using the data in a file to query a directory, or using other information of a file or directory to query a directory or a file. The specific implementation process is similar, that is, a prefix compression index is constructed using a combination of some key information, and then the prefix compression index is queried using the received query statement. Alternatively, an inverted list is constructed using some key information, and then the candidate target is determined after the intersection of the inverted lists of the key information included in the query statement, and the query target is determined from the candidate targets using the key information in the query statement. For details, please refer to the aforementioned content and will not be repeated here.

[0255] In the embodiment of the present application, the query device 100 may be a hardware device, such as a computing device, wherein an acceleration device is deployed in the computing device, and the functions of the query device 100 may be implemented by the processor in the computing device and the acceleration device.

[0256] The hardware components included in the computing device 10 are described below. As shown in Figure 4, which is a schematic diagram of the structure of a computing device 10 provided in an embodiment of the present application, the computing device 10 includes an I / O interface 130, a processor 110, a memory 120, and an acceleration device 150. The I / O interface 130, the processor 110, the memory 120, and the acceleration device 150 can be connected via a system bus, which can be a Peripheral Component Interconnect Express (PCIe) bus, a Compute Express Link (CXL), a Universal Serial Bus (USB) protocol, or other protocols.

[0257] FIG4 exemplarily shows one of the connection modes. In FIG4 , the acceleration device 150 can be directly inserted into a card slot on the motherboard of the computing device 10 and exchange data with the processor 110 through the PCIe bus 140 .

[0258] The I / O interface 130 is used to communicate with devices outside the computing device 10. For example, data (such as the first configuration information, the second configuration information, the query request, the user's metadata or data) sent by the device outside the computing device 10 is received through the I / O interface 130, or a query target is fed back to the device outside the computing device 10 through the I / O interface 130.

[0259] Processor 110 is the computing and control core of computing device 10. It can be a central processing unit (CPU) or other specific integrated circuit. Processor 110 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0260] The memory 120 is typically used to store computer program instructions. The memory 120 can also be used to temporarily store metadata, data, or indexes. The memory 120 has the advantage of fast access speed. The memory 120 typically uses dynamic random access memory (DRAM). In addition to DRAM, the memory 120 can also be other random access memories, such as static random access memory (SRAM), storage class memory (SCM), etc. In addition, the memory 120 can also be read-only memory (ROM). As for read-only memory, for example, it can be programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), etc. The memory 120 can also be a dual in-line memory module or dual-line memory module (DIMM), flash memory (FLASH), hard disk drive (HDD), or solid state drive (SSD).

[0261] The processor 110 is connected to the memory 120 via a double data rate (DDR) bus or other types of buses. The memory 120 is understood to be the internal memory 120 of the computing device 10, and the memory 120 is also called the main memory.

[0262] The processor 110 can execute all or part of the steps performed by the query device 100100 in the embodiment shown in FIG. 2 by calling the computer program instructions in the memory 120 .

[0263] Although not shown, the computing device 10 also includes a persistent memory, or there is a memory that the computing device 10 can access remotely. Regardless of the type of persistent memory, the persistent memory can be used to store data, metadata, or constructed indexes.

[0264] The memory that the computing device 10 can remotely access can be a memory that is connected to the computing device 10 via a network and is located outside the computing device 10. The memory can be a volatile memory, such as RAM, DRAM, SCM, or SRAM. It can also be a non-volatile memory, such as ROM, flash memory, HDD, SSD, or SCM.

[0265] The persistent memory included in the computing device 10 may be connected to the computing device 10 via a system bus. The memory may be a non-volatile memory such as ROM, flash memory, HDD, SSD, etc.

[0266] The acceleration device 150 is connected to the computing device 10. The acceleration device 150 can be an external device of the computing device 10. The acceleration device 150 can also be deployed inside the computing device 10, such as on the motherboard or backplane of the computing device 10. FIG4 is a schematic diagram of the acceleration device 150 deployed inside the computing device 10.

[0267] The acceleration device 150 can serve as a data processing module attached to the computing device 10, assuming some of the functions of the computing device 10. In other words, some of the functions of the computing device 10 are offloaded to the acceleration device 150, which processes data and performs some tasks on behalf of the computing device 10 (such as the processor 110 in the computing device 10), thereby alleviating the pressure on the processor 110 in the computing device 10 and freeing up the computing power of the processor 110.

[0268] The embodiments of the present application do not limit the specific functions undertaken by the acceleration device 150. For example, the acceleration device 150 can carry the query pre-filtering function to determine whether the query request can continue to be processed. If the query request does not pass the query pre-filtering, the acceleration device 150 can directly reject the query request; if the query request passes the query pre-filtering, the acceleration device 150 can forward the query request to the processor 110 of the computing device 10, and the processor 110 continues to process the query request. For another example, the acceleration device 150 can assume the data processing function of the query device 100, complete the metadata processing and index construction (after the acceleration device stores the metadata and constructs the index, it can inform the processor 110 in the computing device 10 of the storage location of the metadata and index). The processor 110 in the computing device 10 assumes the data query function of the query device 100 and processes the received query request.

[0269] For another example, the acceleration device 150 and the processor 110 in the computing device 10 can cooperate to implement the data query function of the query device 100. As for other functions of the query device 100, in this scenario, they can be implemented by the acceleration device 150 and / or the processor 110 in the computing device 10. The embodiment of the present application does not limit the implementation method of the acceleration device 150 and the processor 110 cooperating to implement the data query function. The following is a possible implementation method:

[0270] Implementation method 1: The constructed prefix compression index is a prefix compression index suitable for scenarios where a directory path is used to search for a directory or a file path is used to search for a file. The constructed inverted table is an inverted table suitable for scenarios where data in a file is used to search for a file.

[0271] Then, for a query request containing the path of the query target, the query request is processed by the processor 110 of the computing device 10. The specific processing method can be seen in steps 207 to 209. For a query request containing data in the query target, the query request is processed by the acceleration device 150. The specific processing method can be seen in steps 210 to 213.

[0272] Implementation method 2: The constructed prefix compression index is a prefix compression index suitable for searching for directories using directory paths or searching for files using file paths, and the directory-file combination corresponding to the prefix compression index must include the root directory of the file system. The constructed inverted table is a inverted table suitable for searching for directories using directory paths or searching for files using file paths.

[0273] Then, the computing device 10 can process a query request carrying a path to a query target. If the path to the query target carried in the query request includes a root directory, the query request is processed by the processor 100 of the computing device 10. The specific processing method can be seen in steps 207 to 209. If the path to the query target carried in the query request does not include a root directory, the query request is processed by the acceleration device 150. The specific processing method is similar to the method described in steps 210 to 213. The difference is that the type of query statement carried in the query request is different. The basic processing method is similar: determine the posting list corresponding to each word (i.e., directory name or file name), intersect the posting lists, determine the candidate target, and then determine the query target using the position of each word in the path to the query target and the position of the word recorded in the posting list in the candidate target.

[0274] Implementation method three: the constructed prefix compression index is a prefix compression index suitable for scenarios where data in a file is used to query a file, and the constructed inverted table is an inverted table suitable for scenarios where a directory path is used to search for a directory or a file path is used to search for a file.

[0275] Then, the computing device 10 can process a query request carrying data in the query target. If the data in the query target carried by the query request includes multiple word segments, the query request is processed by the processor 110 of the computing device 10. The specific processing method is similar to steps 207 to 209. The difference is that the type of query statement carried in the query request is different. The basic processing method is similar: determine the target prefix compression index corresponding to the character combination (that is, the character combination composed of one or more words after the word segmentation of the data in the query target) from the prefix compression index. Then, according to the position of each word segment in the data in the query target and the word segment position recorded in the target prefix compression index, determine the first target index item, and then determine the identifier of the query target. The query target is obtained according to the identifier of the query target. If the path of the query target carried by the query request only includes one word segment, the query request is processed by the acceleration device 150. The specific processing method can be found in steps 210 to 213.

[0276] Implementation method 4: The constructed inverted list exists in the form of a skip list and a bitmap. The constructed inverted list may include an inverted list for a scenario where data in a file is used to query a file and / or an inverted list suitable for a scenario where a directory is searched for a directory using a directory path or a file is searched for a file using a file path.

[0277] Assuming a scenario where data in a file is used to query a file, if the shortest length of the postings list corresponding to the words after the data in the query target in the query request is greater than a first threshold, the query request is processed by the acceleration device 150. The specific processing method can be seen in steps 210 to 213, where the acceleration device 150 performs step 211 on the bitmap. If the shortest length of the postings list corresponding to the words after the data in the query target in the query request is less than the first threshold, the query request is processed by the processor 100 of the computing device 10. The specific processing method is similar to steps 210 to 213, where the acceleration device 150 performs step 211 on the skip list.

[0278] Assuming that in a scenario where a directory path is used to search for a directory or a file path is used to search for a file, when the shortest length of the posting list corresponding to the words in the path of the query target in the query request is greater than a first threshold, the query request is processed by the acceleration device 150. The specific processing method can be seen in steps 210 to 213, wherein the acceleration device 150 performs step 211 on the bitmap. When the shortest length of the posting list corresponding to the words after the data in the query target in the query request is divided is not greater than the first threshold, the query request is processed by the processor 110 of the computing device 10. The specific processing method is similar to steps 210 to 213, wherein the acceleration device 150 performs step 211 on the skip list, except that the posting list corresponds to the directory name or file name, rather than to a specific data in the file.

[0279] In addition, in embodiments of the present application, the acceleration device 150 may process the query request in other ways. For example, the acceleration device 150 may store a finite state machine that can be used to determine whether a certain word or words exist in a directory (or file), or whether a certain word or words exist in a directory (or file) path. The acceleration device 150 may use this finite state machine to process the query request.

[0280] The structure of the acceleration device 150 is described below. The acceleration device 150 includes a processor, which can be a data processing unit (DPU) 151, a graphics processing unit (GPU), a tensor processing unit (TPU), or a neural network processing unit (NPU), etc., which has data processing capabilities. In Figure 4, the acceleration device 150 is described as an example in which the processor is DPU 151. Optionally, the acceleration device 150 also includes a memory 152, a power supply circuit, etc. The DPU 151 and the memory 152 are connected via a system bus, which can be a PCIe-based line or a bus using CXL, USB protocol, or other protocols.

[0281] DPU151 is the main computing unit of the acceleration device 150 and the core unit of the acceleration device 150. DPU151 undertakes the main functions of the acceleration device 150.

[0282] In the example shown in FIG4 , the processor and / or acceleration device 150 in the computing device 10 implements the functions of the query device 100. In actual applications, a similar structure can also be applied to the storage device 200. That is, the storage device 200 can also exist in the form of the computing device 10, and the functions of the storage device 200 (such as storing or transmitting data under the instruction of the query device 100) can be implemented by the acceleration device 150.

[0283] Based on the same inventive concept as the method embodiment, the present application also provides a query device for executing the method performed by query device 100 in the method embodiment described above. As shown in Figure 5, query device 500 includes a receiving module 501, an indexing module 502, and a feedback module 503. Specifically, in query device 500, each module is connected via a communication path.

[0284] The receiving module 501 is used to receive a query request, where the query request is used to obtain a query target from a file system. The query request carries a query statement, which includes data in the query target or a path to the query target, and the query statement includes a wildcard.

[0285] Index module 502 is used to: query the prefix compression index based on the query statement to determine the identification of the query target, the prefix compression index records the identification of the file or the identification of the directory, and the file or directory meets any of the following conditions: the path of the file contains a directory-file combination, the path of the directory contains a directory-file combination, the file contains a character combination, and the directory contains a character combination; obtain the query target according to the identification of the query target.

[0286] The directory file combination includes at least one file name and at least one directory name, or includes at least two directory names, and the character combination includes at least two characters.

[0287] The feedback module 503 is used to provide feedback on the query target.

[0288] As a possible implementation, the device further includes a data processing module 504 .

[0289] The receiving module 501 obtains first configuration information, which indicates the types of information included in the metadata. The types of information included in the metadata include some or all of the following: file identification, file path, directory identification, and directory path.

[0290] The data processing module 504 stores metadata of the file or directory based on the first configuration information.

[0291] As a possible implementation, the receiving module 501 obtains second configuration information, where the second configuration information indicates enabling the prefix compression index.

[0292] The data processing module 504 constructs a prefix compression index according to the second configuration information and metadata of the stored files or directories.

[0293] As a possible implementation, the second configuration information also indicates the maximum level of prefix compression index, which describes the total number of directory names and file names in the directory-file combination, or the total number of characters in the character combination.

[0294] As a possible implementation, the prefix compression index includes at least one index item, the index item includes the identifier of the file or the identifier of the directory, and the index item also includes the position of the file name in the file path or the position of the directory name in the directory path.

[0295] As a possible implementation, the prefix compression index includes at least one index item, the index item includes an identifier of a file or a directory, and the index item further includes:

[0296] The position of a character in a character combination in a directory, the position of a character in a character combination in a file.

[0297] As a possible implementation, the index module 502 determines a target prefix compression index from the prefix compression index according to a query statement, wherein the query statement includes a target directory file combination corresponding to the target prefix compression index. Thereafter, the index module 502 determines a first target index item from the target prefix compression index according to the query statement.

[0298] If the first target index includes the identifier of the target file, the identifier of the query target is the identifier of the target file, and for the file name or directory name included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following:

[0299] The first position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target file;

[0300] The second position information included in the first target index item is consistent with the position of the directory name in the query statement, and the second position information is the position of the directory name in the path of the target file.

[0301] If the first target index includes the identifier of the target directory, the identifier of the query target is the identifier of the target directory, and for the file names or directory names included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following:

[0302] The third position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target directory;

[0303] The fourth position information included in the first target index item is consistent with the position of the directory name in the query statement, and the fourth position information is the position of the directory name in the path of the target directory.

[0304] As a possible implementation, the index module 502 may also use a postings table for querying. The index module 502 may query the postings table based on the query statement, determine the postings table corresponding to each word in the query statement, and determine the candidate targets included in the postings table corresponding to each word according to the postings table corresponding to each word. Then, the query target is determined from the candidate targets according to the position of each word in the query statement and the position of the word recorded in the postings table corresponding to each word in the candidate targets.

[0305] As a possible implementation, for a word in a query statement, the query target satisfies:

[0306] The position of the word in the query statement is the same as the word recorded in the query target in the inverted list corresponding to the word.

[0307] As a possible implementation, the indexing module 502 may also determine whether the query statement satisfies some or all of the following:

[0308] The frequency of occurrence of a word in the query statement is not greater than the maximum frequency of occurrence of the word;

[0309] The length of the query statement is not greater than the maximum length of the data in the file or the maximum length of the path;

[0310] The position of the word in the query statement is consistent with the preset position of the word.

[0311] The division of modules in the embodiments of the present application is illustrative and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present application may be integrated into a single processor, or may exist physically separately, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or software functional modules.

[0312] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions for enabling a terminal device (which can be a personal computer, mobile phone, or network device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0313] The present application further provides a computing device 600 as shown in Figure 6. The computing device 600 includes a bus 601, a processor 602, a communication interface 603, and a memory 604. The processor 602, the memory 604, and the communication interface 603 communicate with each other via the bus 601.

[0314] The processor 602 may be a CPU, or other general-purpose processors, DSP, ASIC, FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0315] Memory 604 can be DRAM. In addition to DRAM, memory 604 can also be other random access memories, such as SRAM. Memory 602 can also be ROM. Read-only memory, for example, can be PROM, EPROM, etc. Memory 604 can also be flash memory, HDD, or SSD, etc.

[0316] The memory 604 stores computer program instructions, and the processor 602 executes the computer program instructions to perform the steps performed by the query device 100 in the method described in Figure 2. The memory 604 may also include other software modules required for running processes such as an operating system (such as multiple modules in the query device 500). The operating system may be LINUX TM ,UNIX TM ,WINDOWS TM wait.

[0317] The present application also provides a computing device system, comprising at least one computing device 700 as shown in FIG7 . The computing device 700 comprises a bus 701, a processor 702, a communication interface 703, and a memory 704. The processor 702, the memory 704, and the communication interface 703 communicate with each other via the bus 701. The at least one computing device 700 in the computing device system communicates with each other via a communication path.

[0318] The specific types of the processor 702 and the memory 704 can be found in the description of the processor 602 and the memory 604, which will not be described here. The processor 702 executes the computer program instructions stored in the memory 704 to execute part or all of the steps performed by the detection device 100 in the method described in Figure 2. The memory may also include other software modules required for running processes, such as an operating system. The operating system may be LINUX TM ,UNIX TM ,WINDOWS TM wait.

[0319] At least one computing device 700 in the computing device system establishes communication with each other via a communication network, and each computing device 700 runs any one or multiple modules in the query apparatus 500 .

[0320] The descriptions of the processes corresponding to the above figures have different focuses. For parts that are not described in detail in a certain process, please refer to the relevant descriptions of other processes.

[0321] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes computer program instructions that, when loaded and executed on a computer, fully or partially generate the process or functions described in FIG. 2 according to the embodiment of the present invention.

[0322] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line, or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., an SSD).

[0323] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include these modifications and variations.

Claims

1. A query method, characterized in that, Including: Receiving a query request for obtaining a query target from a file system, where the query request carries a query statement including data in the query target or a path of the query target, and the query statement includes a wildcard; Querying a prefix compression index based on the query statement to determine an identifier of the query target, where the prefix compression index records identifiers of files or directories, and the file or the directory satisfies any one of the following conditions: the path of the file includes a directory file combination, the path of the directory includes the directory file combination, the file includes a character combination, or the directory includes the character combination; Wherein, the directory file combination includes at least one file name and at least one directory name, or includes at least two directory names, and the character combination includes at least two characters; Obtaining and feeding back the query target according to the identifier of the query target.

2. The method according to claim 1, wherein The method further includes: Obtaining first configuration information indicating information types included in metadata, where the information types included in the metadata include some or all of the following: identifier of a file, path of a file, identifier of a directory, path of a directory; Storing the metadata of the file or the directory based on the first configuration information.

3. The method according to claim 2, wherein The method further includes: Obtaining second configuration information indicating enabling the prefix compression index; Constructing the prefix compression index according to the second configuration information and the stored metadata of the file or the directory.

4. The method according to claim 3, wherein The second configuration information further indicates a maximum level of the prefix compression index, and the maximum level of the prefix compression index describes the total number of directory names and file names in the directory file combination, or describes the total number of characters in the character combination.

5. The method according to any one of claims 1 to 4, characterized in that The prefix compression index includes at least one index entry, the index entry includes the identifier of the file or the identifier of the directory, and the index entry further includes the position of the file name of the file in the path of the file, or the position of the directory name of the directory in the path of the directory.

6. The method according to any one of claims 1 to 4, characterized in that The prefix compression index includes at least one index entry, and the index entry includes the identifier of the file or the identifier of the directory, and the index entry further includes: The position of the characters in the character combination in the directory, or the position of the characters in the character combination in the file.

7. The method according to claim 1 or 5, characterized in that, The querying the prefix compression index based on the query statement to determine the identifier of the query target includes: Determining a target prefix compression index from the prefix compression index according to the query statement, where the query statement includes a target directory file combination corresponding to the target prefix compression index; Determining a first target index entry from the target prefix compression index according to the query statement; If the first target index includes the identifier of the target file, the identifier of the query target is the identifier of the target file, and for the file name or directory name included in the target directory file combination, the first target index entry and the query statement satisfy some or all of the following: The first position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target file; The second position information included in the first target index item is consistent with the position of the directory name in the query statement, and the second position information is the position of the directory name in the path of the target file; If the identifier of the target directory is included in the first target index, the identifier of the query target is the identifier of the target directory. For the file name or directory name included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following: The third position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target directory; The fourth position information included in the first target index item is consistent with the position of the directory name in the query statement, and the fourth position information is the position of the directory name in the path of the target directory.

8. The method according to claim 1, wherein Querying the prefix compression index based on the query statement of the query target to determine the identifier of the query target includes: Querying the inverted list based on the query statement to determine the inverted list corresponding to each word in the query statement; Determining the candidate targets included in all the inverted lists corresponding to each word according to the inverted lists corresponding to each word; Determining the query target from the candidate targets according to the position of each word in the query statement and the position of the word recorded in the inverted list corresponding to each word in the candidate target; Feedbacking the query target to the user.

9. The method according to claim 8, wherein The according to the position of each word in the query statement and the position of the word recorded in the inverted list corresponding to each word in the candidate target includes: For a word in the query statement, the query target satisfies: The position of the word in the query statement is the same as the position of the word recorded in the inverted list corresponding to the word in the query target.

10. The method according to any one of claims 1 to 9, characterized in that, Before querying the prefix compression index based on the query statement of the query target to determine the identifier of the query target, it further includes: Determining that the query statement satisfies some or all of the following: The occurrence frequency of the word in the query statement is not greater than the maximum occurrence frequency of the word; The length of the query statement is not greater than the maximum length of the data in the file or the maximum length of the path; The position of the word in the query statement is consistent with the preset position of the word.

11. A query device, characterized in that, Including: A receiving module, configured to: receive a query request for obtaining a query target from a file system, where the query request carries a query statement, the query statement includes data in the query target or the path of the query target, and the query statement includes wildcards; An index module, configured to: query a prefix-compressed index based on the query statement to determine an identifier of the query target, where the prefix-compressed index records identifiers of files or directories, and the file or the directory satisfies any of the following conditions: the path of the file contains a directory file combination, the path of the directory contains the directory file combination, the file contains a character combination, or the directory contains the character combination; obtain the query target according to the identifier of the query target. Wherein, the directory file combination includes at least one file name and at least one directory name, or includes at least two directory names, and the character combination includes at least two characters. A feedback module, configured to: feedback the query target.

12. The device according to claim 11, characterized in that The apparatus further includes a data processing module. The receiving module is further configured to: obtain first configuration information, where the first configuration information indicates information types included in the metadata, and the information types included in the metadata include some or all of the following: identifier of a file, path of a file, identifier of a directory, path of a directory. The data processing module is configured to: store the metadata of the file or the directory based on the first configuration information.

13. The apparatus according to claim 12, wherein The receiving module is configured to: obtain second configuration information, where the second configuration information indicates enabling the prefix-compressed index. The data processing module is configured to: construct the prefix-compressed index according to the second configuration information and the stored metadata of the file or the directory.

14. The device according to claim 13, characterized in that, The second configuration information further indicates a maximum level of the prefix-compressed index, and the maximum level of the prefix-compressed index describes the total number of directory names and file names in the directory file combination, or describes the total number of characters in the character combination.

15. The device according to any one of claims 11 to 14, characterized in that, The prefix-compressed index includes at least one index entry, the index entry includes the identifier of the file or the identifier of the directory, and the index entry further includes the position of the file name of the file in the path of the file, or the position of the directory name of the directory in the path of the directory.

16. The device according to any one of claims 11 to 14, characterized in that, The prefix-compressed index includes at least one index entry, and the index entry includes the identifier of the file or the identifier of the directory, and the index entry further includes: The position of the characters in the character combination in the directory, or the position of the characters in the character combination in the file.

17. The device according to claim 11 or 15, characterized in that, The index module is configured to: Determine a target prefix-compressed index from the prefix-compressed index according to the query statement, where the query statement includes a target directory file combination corresponding to the target prefix-compressed index. Determine a first target index entry from the target prefix-compressed index according to the query statement. If the first target index includes the identifier of the target file, the identifier of the query target is the identifier of the target file, and for the file name or directory name included in the target directory file combination, the first target index entry and the query statement satisfy some or all of the following: The first position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target file; The second position information included in the first target index item is consistent with the position of the directory name in the query statement, and the second position information is the position of the directory name in the path of the target file; If the identifier of the target directory is included in the first target index, the identifier of the query target is the identifier of the target directory. For the file name or directory name included in the target directory file combination, the first target index item and the query statement satisfy some or all of the following: The third position information included in the first target index item is consistent with the position of the file name in the query statement, and the first position information is the position of the file name in the path of the target directory; The fourth position information included in the first target index item is consistent with the position of the directory name in the query statement, and the fourth position information is the position of the directory name in the path of the target directory.

18. The device according to claim 11, characterized in that, The indexing module is used for: Querying the inverted list based on the query statement to determine the inverted lists corresponding to the words in the query statement; Determining candidate targets included in all the inverted lists corresponding to the words according to the inverted lists corresponding to the words; Determining the query target from the candidate targets according to the positions of the words in the query statement and the positions of the words in the candidate targets recorded in the inverted lists corresponding to the words.

19. The device according to claim 18, characterized in that, For a word in the query statement, the query target satisfies: The position of the word in the query statement is the same as the position of the word in the query target recorded in the inverted list corresponding to the word.

20. The device according to any one of claims 11 to 19, characterized in that, The indexing module is further used for: Determining that the query statement satisfies some or all of the following: The occurrence frequency of the word in the query statement is not greater than the maximum occurrence frequency of the word; The length of the query statement is not greater than the maximum length of the data in the file or the maximum length of the path; The position of the word in the query statement is consistent with the preset position of the word.

21. A computing device, characterized in that, The computing device includes a processor and a memory; The memory is used for storing computer program instructions; The processor executes by calling the computer program instructions stored in the memory to execute the method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by the computing device, the computing device executes the method according to any one of claims 1 to 10 above.

Images