A data processing method and device, electronic equipment and computer storage medium

Abstract

The application provides a data processing method and device, electronic equipment and computer storage medium, comprising traversing each element of a path in a query request; if the current traversed element is a field and the last traversed element is a field, combining the data corresponding to the field based on the first data table and the query result of the last traversed element as the query result of the element; if the current traversed element is a field and the last traversed element is a character, querying the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result; if it is determined that the current traversed element is a character, it is determined that the query result of the last traversed element exists in the second data table, and the query result of the last traversed element is taken as the query result of the character; and then the metadata corresponding to the path is determined. The application can improve the query efficiency by querying data through the first data table and the second data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a data processing method and device, electronic equipment and computer storage medium. BACKGROUND

[0002] In a distributed file system HDFS, whether it is a directory or a file, it is regarded as an INode node in the file system directory tree, and if there is a subdirectory or a file under the directory, the INode reference of the subdirectory or the file will be saved in the children set. HDFS maintains the directory structure of the entire file system in this way. After NameNode loads FsImage into the memory, a directory tree structure is formed. When data query is performed, each content in the directory tree structure needs to be traversed, thereby causing slow query speed. SUMMARY

[0003] Therefore, the embodiments of the present application provide a data processing method and device, electronic equipment and computer storage medium to solve the problem of slow query speed in the prior art.

[0004] To achieve the above object, the embodiments of the present application provide the following technical solutions.

[0005] The first aspect of the embodiments of the present application shows a data processing method, which comprises:

[0006] receiving a query request;

[0007] if there are N elements in the path of the query request, traversing each element of the path in the query request, N being a positive integer greater than or equal to 2;

[0008] if the current traversed element is a field and the last traversed element is a field, combining the query result of the last traversed element and the data corresponding to the field based on the first data table as the query result corresponding to the element;

[0009] if the current traversed element is a field and the last traversed element is a character, querying a second data table based on the query result of the last traversed element and the field to obtain the corresponding query result;

[0010] if it is determined that the current traversed element is a character, determining whether the query result of the last traversed element exists in the second data table, if it exists, taking the query result of the last traversed element as the query result of the character, and the current traversed element is not the first element in the path;

[0011] If the query result corresponding to the Nth traversed element is determined from the first data table, metadata of the path is determined based on the query result corresponding to the Nth traversed element.

[0012] Optionally, if the current traversed element is the first element in the path, it is determined whether the first data table has the first element.

[0013] If the first element exists, data corresponding to the first element is obtained from the first data table.

[0014] The data corresponding to the first element is taken as the query result of the element.

[0015] Optionally, the method further comprises:

[0016] If the query result corresponding to the Nth traversed element is determined from the second data table, data corresponding to the query result is obtained from the first data table.

[0017] The data corresponding to the query result is taken as the metadata of the path.

[0018] Optionally, the method further comprises:

[0019] Setting a target identifier of a node in a file system directory tree;

[0020] Based on the file system directory tree, an association relationship of the node is determined.

[0021] Based on the target identifier and the association relationship of the file, a first data table is generated.

[0022] Optionally, the method further comprises:

[0023] Based on the association relationship of the node and the target identifier of the file corresponding to each node, a second data table is generated.

[0024] The second aspect of the embodiment of the application shows a data processing apparatus, a receiving unit is configured to receive a query request.

[0025] The processing unit is configured to traverse each element of the path in the query request if there are N elements in the path of the query request, N being a positive integer greater than or equal to 2; if the current traversed element is a field and the last traversed element is a field, combining the data corresponding to the field and the query result of the last traversed element as the query result corresponding to the element based on the first data table; if the current traversed element is a field and the last traversed element is a character, querying a second data table based on the query result of the last traversed element and the field to obtain a corresponding query result; if it is determined that the current traversed element is a character, determining whether the query result of the last traversed element exists in the second data table, and if so, taking the query result of the last traversed element as the query result of the character, the current traversed element not being the first element in the path; and if the query result corresponding to the Nth traversed element is determined from the first data table, determining the metadata of the path based on the query result corresponding to the Nth traversed element.

[0026] Optionally, the method further comprises:

[0027] If the current traversed element is the first element in the path, determining whether the first element exists in the first data table.

[0028] If so, obtaining the data corresponding to the first element from the first data table.

[0029] Taking the data corresponding to the first element as the query result corresponding to the element.

[0030] Optionally, the processing unit is further configured to:

[0031] If the query result corresponding to the Nth traversed element is determined from the second data table, obtaining the data corresponding to the query result from the first data table.

[0032] Taking the data corresponding to the query result as the metadata of the path.

[0033] The third aspect of the embodiment of the present application shows an electronic device, which is configured to run a program, wherein the program performs the data processing method shown in the first aspect of the embodiment of the present application when running.

[0034] The fourth aspect of the embodiment of the present application shows a computer storage medium, which comprises a program, wherein the program controls the device where the storage medium is located to perform the data processing method shown in the first aspect of the embodiment of the present application when running.

[0035] Based on the data processing method, device, electronic equipment and computer storage medium provided by the embodiment of the application, the method comprises: receiving a query request; if there are N elements in the path of the query request, traversing each element of the path in the query request, N being a positive integer greater than or equal to 2; if the current traversed element is a field and the last traversed element is a field, combining the query result of the last traversed element and the data corresponding to the field based on the first data table as the query result corresponding to the element; if the current traversed element is a field and the last traversed element is a character, querying the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result; if it is determined that the current traversed element is a character, determining whether the query result of the last traversed element exists in the second data table, if it exists, taking the query result of the last traversed element as the query result of the character, and the current traversed element is not the first element in the path; if the query result corresponding to the Nth traversed element is determined from the first data table, determining the metadata of the path based on the query result corresponding to the Nth traversed element. After the first data table and the second data table are constructed, the data is queried through the first data table and the second data, so that different execution steps are triggered according to different elements in the path, thereby improving the efficiency of data query. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0037] Figure 1 The flowchart for generating the first data table shown in the embodiment of the present application;

[0038] Figure 2 The flowchart for generating the data table shown in the embodiment of the present application;

[0039] Figure 3 The flowchart of the data processing method shown in the embodiment of the present application;

[0040] Figure 4 The structural diagram of the data processing device shown in the embodiment of the present application;

[0041] Figure 5 The structural diagram of another data processing device shown in the embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be apparently and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0043] The terms "first", "second", "third", "fourth" and the like in the description and claims of the present application and in the above drawings (if any) are used to distinguish similar objects, and do not necessarily indicate a particular order or sequence. It should be understood that the data thus used can be interchanged, where appropriate, so that the embodiments described herein can be carried out in a different order than the one illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to the process, method, product or device.

[0044] It should be noted that the description involving "first", "second" and the like in the present application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the fact that a person of ordinary skill in the art can realize it. When the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist, and is not within the protection scope required by the present application.

[0045] In the present application, the terms "including", "containing" or any other variants thereof are intended to cover non-exclusive inclusions, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the sentence "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0046] As known from the background art, in the HDFS, whether it is a directory or a file, it is regarded as an INode node in the file system directory tree, and if there is a subdirectory or a file under the directory, the INode reference of the subdirectory or the file will be saved in the children set. The HDFS maintains the directory structure of the entire file system in this way. After the NameNode loads the FsImage into the memory, a directory tree structure is formed. However, using the directory tree method, the entire directory tree information needs to be loaded into the memory, which will occupy a large amount of memory; and when data query is performed, each content in the directory tree needs to be traversed, thereby causing slow query speed.

[0047] In order to better understand the terms shown in the embodiments of the present application, the related terms are explained as follows.

[0048] The NameNode is a manager in the HDFS, and is mainly responsible for the namespace of the file system, the cluster configuration information, and the replication of data blocks, etc.

[0049] The metadata is the reference relationship of each file and each data block in the file system saved in the memory by the NameNode.

[0050] The Bloom filter is a long binary vector and a series of random mapping functions. The Bloom filter can be used to query whether an element is in a set.

[0051] Referring to Figure 1 The flowchart for generating the first data table shown in the embodiments of the present application is shown, and the method comprises the following steps:

[0052] Step S101: setting the target identifier of a node in the file system directory tree.

[0053] In the process of implementing step S101, the target identifier of the file corresponding to each node in the file system directory tree is set according to a random algorithm, that is, the ID number inode id of the node.

[0054] Step S102: determining the association relationship of the node based on the file system directory tree.

[0055] In the process of implementing step S102, for each node, the association relationship between the nodes is described through the target identifier of the node.

[0056] Step S103: generating a first data table based on the association relationship of the node, the target identifier of the node, and the file corresponding to the target identifier.

[0057] It should be noted that the implementation process of step S103 comprises the following steps:

[0058] Step S11: taking the target identifier as a key of a key-value pair.

[0059] Step S12: determining first data in a file corresponding to the target identifier based on the association relationship of the node.

[0060] It should be noted that the first data only includes the first identifier having a parent-child relationship with the node, the name of the node corresponding to the first identifier, and whether the name of the node corresponding to the first identifier exists in the directory structure.

[0061] In the process of implementing step S12, for each node corresponding to a target identifier, the association relationship of the node is traversed to determine the first identifier having a parent-child relationship with the node; the name of the node corresponding to the target identifier is obtained from the file corresponding to the target identifier; and then it is determined whether the name exists in the directory structure.

[0062] Step S13: taking the first data as a value of the key-value pair.

[0063] Step S14: constructing a first data table based on the key of the key-value pair and the value of the key-value pair.

[0064] In the process of implementing step S14, a first data table is constructed based on the key of each node and the value corresponding thereto.

[0065] That is, the first data table is a node table, the key of the node table is the identifier inode id of the node, and the value Value is the data inode of the node, which specifically can include the metadata of the file or directory related to the node.

[0066] For example, there are four nodes in the file system tree, and the target identifiers corresponding to the nodes can be id0, id1, id2 and id3. For the first node, the ID of the node is id0, the associated relationship of the node id0 is traversed, the first identifier ID in the parent-child relationship with the node is determined to be -1, the name of the target identifier corresponding node is obtained from the file corresponding to the target identifier id0, and the name is “ / ”. It is determined that the name exists in the directory structure, and the data is is_directory:true. For the second node, the ID of the node is id1, the associated relationship of the node id1 is traversed, the first identifier ID in the parent-child relationship with the node is determined to be id1, the name of the target identifier corresponding node is obtained from the file corresponding to the target identifier id1, and the name is “dir”. It is determined that the name exists in the directory structure, and the data is is_directory:true. For the third node, the ID of the node is id2, the associated relationship of the node id2 is traversed, the first identifier ID in the parent-child relationship with the node is determined to be id2, the name of the target identifier corresponding node is obtained from the file corresponding to the target identifier id2, and the name is “file_a”. It is determined that the name exists in the directory structure, and the data is is_directory:false. For the fourth node, the ID of the node is id3, the associated relationship of the node id3 is traversed, the first identifier ID in the parent-child relationship with the node is determined to be id3, the name of the target identifier corresponding node is obtained from the file corresponding to the target identifier id3, and the name is “file_b”. It is determined that the name exists in the directory structure, and the data is is_directory:false, as shown in Table (1).

[0067] Table (1):

[0068] Key Value id0 inode {id: id0, parent_id: -1, name: " / ", is_directory: true,...} id1 inode {id: id1, parent_id: id1, name: "dir", is_directory: true,...} id2 inode {id: id2, parent_id: id2, name: "file_a", is_directory: false,...} id3 inode {id: id3, parent_id: id3, name: "file_b", is_directory: false,...}

[0069] In the embodiment of the application, the first data table and the second data table are used to replace the traditional directory tree structure in the key-value pair data structure to store the HDFS directory tree information, so as to reduce the memory. The key-value pair database can be used to optimize the file search function. This processing method can reduce the memory occupation and still has high query efficiency. When data query is performed, data query is performed through the first data table and the second data, different execution steps are triggered according to different elements in the path, and the efficiency of data query can be improved.

[0070] Based on the flowchart of the data processing method shown in the above embodiment of the application, another flowchart of a data processing method is also disclosed in the embodiment of the application, as shown in Figure 2 The method comprises the following steps.

[0071] Step S201: setting a target identifier of a node in a file system directory tree.

[0072] In the implementation of step S201, the target identifier of each node corresponding to the file in the file system directory tree is set according to a random algorithm or the like, that is, the inode id of the node.

[0073] Step S202: determining the association relationship of the node based on the file system directory tree.

[0074] In the implementation of step S202, for each node, the association relationship between the nodes is described through the target identifier of the node.

[0075] Step S203: generating a first data table based on the association relationship of the node, the target identifier of the node, and the file corresponding to the target identifier.

[0076] It should be noted that the implementation processes of steps S201 to S203 are the same and can be mutually referenced.

[0077] Step S204: generating a second data table based on the association relationship of the node and the target identifier of each node corresponding to the file.

[0078] It should be noted that the implementation process of step S204 of generating a second data table based on the association relationship of the node and the target identifier of each node corresponding to the file includes the following steps:

[0079] Step S21: taking the target identifier of the node and the name of the node as the key of the key-value pair.

[0080] In the implementation of step S21, the name of the node corresponding to the target identifier is obtained from the file corresponding to the target identifier, and the target identifier and the name of the node are taken as the key of the key-value pair.

[0081] Step S22: determining the second identifier associated with the node based on the association relationship of the node.

[0082] In the implementation of step S22, the association relationship of the node is traversed to determine other nodes having a parent-child relationship with the node, that is, the node is a parent node and the other node is a child node, and the target identifier of the other node is taken as the second identifier.

[0083] Step S23: taking the second identifier as the value of the key-value pair.

[0084] Step S24: constructing the second data table by taking the key of the key-value pair and the value of the key-value pair.

[0085] In the process of implementing step S24, the second data table is constructed for the key of each node and the corresponding value.

[0086] That is, the second data table is an edge table, the key Key in the edge table is "target identifier inodeid of parent node, name of subdirectory / file", and the value Value is the target identifier inode id of the subnode.

[0087] For example, the edge table a is shown in Table (2). The target identifier of the first node in Table (2) is id0, and the name is "dir". The target identifier id0 of the node corresponds to the target identifier id1 of the subnode. id0, "dir" is taken as the key, and the id1 is taken as the value. The target identifier of the second node is id1, and the name is "file_a". The target identifier id1 of the node corresponds to the target identifier id2 of the subnode. id1, "file_a" is taken as the key, and the id1 is taken as the value. The target identifier of the third node is id1, and the name is "file_b". The target identifier id1 of the node corresponds to the target identifier id3 of the subnode. id1, "file_b" is taken as the key, and the id3 is taken as the value.

[0088] Table (2):

[0089] Key Value id0, "dir" id1 id1, "file_a" id2 id1, "file_b" id3

[0090] Optionally, when the second data table is constructed, the prefix in each key, that is, the key is range scanned to maintain a Bloom filter for the prefix of each second data table key, that is, the target identifier, which is used to quickly judge whether the target identifier is in the second data table.

[0091] In the embodiment of the application, the first data table and the second data table are used to replace the traditional directory tree structure in the key-value pair data structure to store the HDFS directory tree information, so as to reduce the memory. The key-value pair database can be used to optimize the file searching function. This processing manner can reduce the memory occupation, and the query efficiency is still high. When data query is performed, the data query is performed through the first data table and the second data table, different execution steps are triggered according to different elements in the path, and therefore the efficiency of data query can be improved.

[0092] Based on the flowchart of the data processing method shown in the embodiment of the application, another flowchart of a data processing method is also disclosed in the embodiment of the application, as shown in Figure 3 The method comprises the following steps.

[0093] Step S301: receiving a query request.

[0094] In the implementation of step S301, the query request input by the user is received.

[0095] It should be noted that the path of the query request includes at least one element, which can be a field or a character.

[0096] In step S302, if there are N elements in the path of the query request, each element in the path is traversed.

[0097] In step S302, N is a positive integer greater than or equal to 2.

[0098] In the implementation of step S302, if there are N elements in the path of the query request, each element is traversed in the order from left to right according to the path.

[0099] It should be noted that the element can be a character or a field, and the path is composed of characters and / or fields.

[0100] The character is a character that a skilled person can obtain through experience or multiple experiments, for example, the first element is a character " / ", which indicates that there is a root node, and the third element is a character " / ", which indicates that there is a child node.

[0101] Optionally, if there is one element in the path of the query request, it is determined whether the first data table includes the element; if so, the data corresponding to the element is obtained from the first data table; and the data corresponding to the element is taken as the metadata corresponding to the path.

[0102] Optionally, if the currently traversed element is the first element in the path, it is determined whether the first data table includes the first element; if so, the data corresponding to the first element is obtained from the first data table; and the data corresponding to the first element is taken as the query result corresponding to the element.

[0103] It should be noted that the first element can be a character or a field.

[0104] In the implementation, it is determined whether the first data table includes the first element, if so, the identifier corresponding to the first element, i.e., the data, is obtained from the first data table, and taken as the query result of the element.

[0105] Optionally, if the first element is a character, it is further determined whether the second data table includes data with the query result as a key.

[0106] Step S303: It is determined whether the currently traversed element is a field, if so, step S304 is executed, if not, it is determined that the currently traversed element is a character, and step S307 is executed.

[0107] It should be noted that the current traversed element is not the first element in the path.

[0108] Step S304: judging whether the last traversed element is a field, if yes, executing step S305, if not, determining that the last traversed element is a character, and then executing step S306.

[0109] In the application, the current traversed element and the last traversed element are used to determine the next step to be executed in steps S303 and S304.

[0110] Step S305: combining the data corresponding to the field and the query result of the last traversed element as the query result of the element based on the first data table.

[0111] It should be noted that the process of implementing step S305 includes the following steps:

[0112] Step S31: determining whether the field exists in the first data table, if yes, executing step S32, if not, outputting information that the path does not exist.

[0113] In the process of implementing step S31, the first data table is traversed to determine whether the field exists in the first data table, if yes, executing step S303, if not, outputting information that the path does not exist.

[0114] Step S32: obtaining the data corresponding to the field from the first data table.

[0115] In the process of implementing step S32, the target identifier corresponding to the field is obtained from the first data table as the data corresponding to the field.

[0116] Step S33: combining the data corresponding to the field and the query result of the last traversed element as the query result of the element.

[0117] Step S306: querying the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result.

[0118] In the process of implementing step S306, since the last traversed element is a character, it can be determined that the field is a child node, and therefore it is necessary to find the data corresponding to the last traversed element and the field from the second data table, if found, the corresponding data is used as the query result of the element.

[0119] Step S307: Determine whether the query result of the element traversed last time exists in the second data table. If yes, execute step S308. If no, output information that the path does not exist.

[0120] In the implementation of step S307, the Bloom filter is used to determine whether the query result of the element traversed last time exists in the second data table as the value of the key field. If yes, execute step S308. If no, output information that the path does not exist.

[0121] Step S308: Take the query result of the element traversed last time as the query result of the character.

[0122] In step S308, the element traversed currently is not the first element in the path.

[0123] In the implementation of step S308, if it is determined that the element traversed currently is a character, it is determined that the query result of the element traversed last time exists in the subnode, and thus the query result of the element traversed last time is taken as the query result of the character.

[0124] Step S309: Determine whether the query result corresponding to the element traversed Nth time is determined from the first data table or the second data table. If the query result corresponding to the element traversed Nth time is determined from the first data table, execute step S310. If the query result corresponding to the element traversed Nth time is determined from the second data table, execute step S311.

[0125] Since the last element in the path can be a subnode of the query result of the element traversed last time or a root node, it is necessary to determine whether the data corresponding to the last element is obtained from the first data table or the second data table, that is, execute step S309.

[0126] Step S310: Determine the metadata of the path based on the query result corresponding to the element traversed Nth time.

[0127] In the implementation of step S310, since the query result corresponding to the last element in the path is determined from the first data table, that is, the metadata is stored in the first data table, the value of the query result in the first data table can be determined as the metadata of the path.

[0128] Step S311: Obtain the data corresponding to the query result from the first data table.

[0129] Step S312: Take the data corresponding to the query result as the metadata of the path.

[0130] In the process of implementing steps S311 to S312, the query result is taken as a key to determine whether the key exists in the first data table, and if so, the value corresponding to the key, i.e., data, is obtained, which is taken as the metadata of the path.

[0131] For example, the query request is to find the metadata of the path " / dir / file_a" node, and the specific steps are as follows: traversing each element in the path; determining that the first element is the character " / "; when it is determined that the character " / " exists in the first data table, the third data id0 corresponding to the character " / " is obtained; using the Bloom filter to determine whether the value with the third data id0 as the key exists in the second data table; if so, it is determined that the second element is the field "dir"; whether the value with "id0, dir" as the key exists in the second data table is found, and if so, the Value value corresponding thereto is obtained as id1; when it is determined that the third element is the character " / ", the Bloom filter is used to determine whether the value with id1 as the key exists in the edge table, and if so, it is determined that the third element is the field "file_a", and whether the value with "id1, file_a" as the key exists in the second data table is found, and if so, the Value value corresponding thereto is obtained as id2; the first data table is found based on the id2, and the metadata corresponding to the id2, i.e., the first data, is determined.

[0132] In the embodiment of the application, the first data table and the second data table are used to replace the traditional directory tree structure in the key-value pair data structure to store the HDFS directory tree information, so as to reduce the memory. The key-value pair database can be used to optimize the file finding function. This processing manner can reduce the memory occupation and the query efficiency is still high. When data query is performed, the first data table and the second data are used for data query, different execution steps are triggered according to different elements in the path, and thus the efficiency of data query can be improved.

[0133] Based on the data processing method shown in the above embodiment of the application, a structure diagram of a data processing device is also shown in the embodiment of the application, as shown in Figure 4 The device includes:

[0134] The receiving unit 401 is configured to receive a query request.

[0135] The processing unit 402 is configured to traverse each element of the path in the query request if there are N elements in the path of the query request, N being a positive integer greater than or equal to 2; if the current traversed element is a field and the last traversed element is a field, combining the query result of the last traversed element and the data corresponding to the field in the first data table as the query result corresponding to the element; if the current traversed element is a field and the last traversed element is a character, querying the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result; if it is determined that the current traversed element is a character, determining whether the query result of the last traversed element exists in the second data table, and if so, taking the query result of the last traversed element as the query result of the character, the current traversed element not being the first element in the path; and if the query result corresponding to the Nth traversed element is determined from the first data table, determining the metadata of the path based on the query result corresponding to the Nth traversed element.

[0136] It should be noted that the specific principles and execution processes of each unit in the data processing apparatus disclosed in the embodiments of the present application are the same as those of the data processing method disclosed in the embodiments of the present application, and can be referred to the corresponding parts of the data processing method disclosed in the embodiments of the present application, which will not be described here.

[0137] In the embodiments of the present application, the first data table and the second data table are used to replace the traditional directory tree structure in the key-value pair data structure to store the HDFS directory tree information, so as to reduce the memory. The key-value pair database can be used to optimize the file searching function. This processing method can reduce the memory occupation and still has high query efficiency. In order to improve the data query efficiency, the first data table and the second data table are used to perform data query according to the triggering of different elements in the path, so as to trigger different execution steps.

[0138] Optionally, based on the data processing apparatus disclosed in the embodiments of the present application, the processing unit 402 is further configured to:

[0139] If the current traversed element is the first element in the path, it is determined whether the first element exists in the first data table;

[0140] If so, the data corresponding to the first element is obtained from the first data table;

[0141] The data corresponding to the first element is taken as the query result corresponding to the element.

[0142] Optionally, based on the data processing apparatus disclosed in the embodiments of the present application, the processing unit 402 is further configured to:

[0143] if the query result corresponding to the Nth traversed element is determined from the second data table, obtaining data corresponding to the query result from the first data table;

[0144] taking the data corresponding to the query result as the metadata of the path.

[0145] Optionally, based on the data processing apparatus shown in the above embodiments of the present application, in combination with Figure 4 , see Figure 5 Further comprising a construction unit 403.

[0146] The construction unit is configured to set a target identifier of a node in a file system directory tree, determine an association relationship of the node based on the file system directory tree, and generate a first data table based on the target identifier and the association relationship of the file.

[0147] Optionally, the construction unit 403 is further configured to generate a second data table based on the association relationship of the node and the target identifier of the file corresponding to each node.

[0148] The embodiments of the present application further disclose an electronic device configured to run a database stored procedure, wherein the database stored procedure is configured to perform the above Figures 1 to 3 disclosed data processing method.

[0149] The embodiments of the present application further disclose a computer storage medium, wherein the storage medium comprises a database stored procedure, and wherein the database stored procedure is configured to control a device where the storage medium is located to perform the above Figures 1 to 3 disclosed data processing method.

[0150] In the context of the present disclosure, the computer storage medium can be a tangible medium, which can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium can include one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disks read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0151] The various embodiments described in this specification are described in progressive order of complexity, where simpler embodiments are built to more complex embodiments; each embodiment builds upon the previous embodiment(s), and each embodiment is used to describe differences between embodiments. In particular, the system or system embodiments are described with less detail as they are substantially similar to the method embodiments. The systems and system embodiments described above are merely illustrative and are not meant to limit the scope of the application as the components pertaining to the separate units as described can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located at one place, or distributed on a plurality of network units. Depending upon the actual requirements, a part or all of the modules can be implemented to achieve the objectives of the embodiments of the present application. One of ordinary skill in the art can understand and implement the embodiments without undue creative labor.

[0152] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or both. To clearly illustrate the interchangeability of hardware and software, and to avoid obscuring the disclosure, the aforementioned examples have been described in general terms that are applicable to both hardware and software implementations. The specific implementation details are left to the discretion of the skilled person. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0153] The various embodiments described in this specification are described in progressive order of complexity, where simpler embodiments are built to more complex embodiments; each embodiment builds upon the previous embodiment(s), and each embodiment is used to describe differences between embodiments. In particular, the system or system embodiments are described with less detail as they are substantially similar to the method embodiments. The systems and system embodiments described above are merely illustrative and are not meant to limit the scope of the application as the components pertaining to the separate units as described can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located at one place, or distributed on a plurality of network units. Depending upon the actual requirements, a part or all of the modules can be implemented to achieve the objectives of the embodiments of the present application. One of ordinary skill in the art can understand and implement the embodiments without undue creative labor.

[0154] Those skilled in the art will further realize that the mechanisms of the various examples described herein are capable of being implemented using any number of combinations of the described features. Accordingly, these examples are not limited to the mechanisms described herein, but rather, the intent is to cover all modifications and alternatives equivalent thereto. The preceding description of the examples is illustrative, and not restrictive. Many other examples will be apparent to those of skill in the art upon reviewing the above description. The scope of the examples should, therefore, be determined not with reference to the above description, but instead should be given to the appended claims, along with their full scope of equivalents.

[0155] The above description of disclosed examples is intended to be illustrative, and not restrictive. Many other examples will be apparent to those of skill in the art upon reviewing the above description. The scope of the examples should, therefore, be determined not with reference to the above description, but instead should be given to the appended claims, along with their full scope of equivalents.

Claims

1. A data processing method, characterized by, The method comprises: receiving a query request; if there are N elements in the path of the query request, traversing each element of the path in the query request, N being a positive integer greater than or equal to 2; if the current traversed element is a field and the last traversed element is a field, combining the query result of the last traversed element and the data corresponding to the field in the first data table as the query result corresponding to the element; if the current traversed element is a field and the last traversed element is a character, querying the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result; if it is determined that the current traversed element is a character, determining whether the query result of the last traversed element exists in the second data table, if it exists, taking the query result of the last traversed element as the query result of the character, and the current traversed element is not the first element in the path; if the query result corresponding to the Nth traversed element is determined from the first data table, determining the metadata of the path based on the query result corresponding to the Nth traversed element; if the query result corresponding to the Nth traversed element is determined from the second data table, obtaining the data corresponding to the query result from the first data table; taking the data corresponding to the query result as the metadata of the path.

2. The method of claim 1, wherein, Further comprising: if the current traversed element is the first element in the path, determining whether the first element exists in the first data table; if it exists, obtaining the data corresponding to the first element from the first data table; taking the data corresponding to the first element as the query result corresponding to the element.

3. The method of claim 1, wherein, Further comprising: setting the target identifier of the node in the file system directory tree; determining the association relationship of the node based on the file system directory tree; generating the first data table based on the association relationship of the target identifier and the file.

4. The method of claim 3, wherein, Further comprising: generating the second data table based on the association relationship of the node and the target identifier of the file corresponding to each node.

5. A data processing apparatus, characterized by, The device comprises: a receiving unit for receiving a query request; The processing unit is configured to traverse each element of the path in the query request if there are N elements in the path of the query request, N being a positive integer greater than or equal to 2; if the current traversed element is a field and the last traversed element is a field, combine the data corresponding to the field and the query result of the last traversed element as the query result corresponding to the element based on the first data table; if the current traversed element is a field and the last traversed element is a character, query the second data table based on the query result of the last traversed element and the field to obtain the corresponding query result; if it is determined that the current traversed element is a character, determine whether the query result of the last traversed element exists in the second data table, and if so, take the query result of the last traversed element as the query result of the character, the current traversed element not being the first element in the path; if the query result corresponding to the Nth traversed element is determined from the first data table, determine the metadata of the path based on the query result corresponding to the Nth traversed element; if the query result corresponding to the Nth traversed element is determined from the second data table, obtain the data corresponding to the query result from the first data table; and take the data corresponding to the query result as the metadata of the path.

6. The apparatus of claim 5, wherein, The processing unit is further configured to: if the current traversed element is the first element in the path, determine whether the first element exists in the first data table; if so, obtain the data corresponding to the first element from the first data table; and take the data corresponding to the first element as the query result corresponding to the element.

7. An electronic device, comprising: The electronic device is configured to run a program, wherein the program performs the data processing method according to any one of claims 1-4 when running.

8. A computer storage medium, characterized in that The storage medium comprises a program, wherein the program controls the device where the storage medium is located to perform the data processing method according to any one of claims 1-4 when running.

Images