Data processing method and device, readable storage medium and program product

Abstract

Embodiments of the invention provide a data processing method and device, a readable storage medium and a program product. The method comprises the steps of obtaining tree structure data; constructing a memory data structure of a node graph based on a father-child node association relationship in the tree structure data and the data of each node; wherein the memory data structure comprises a node array, a key-identifier mapping table and a child node mapping table, and the node array represents an association relationship between a node identity identifier and node data; the key-identifier mapping table represents an association relationship between a key of the node and an identity identifier of the node; the child node mapping table represents an association relationship between an identifier of a father node and a child node list corresponding to the father node, and the child node list comprises an identity identifier of at least one direct child node; and storing the memory data structure of the node graph. Query efficiency can be improved, and node updating complexity can be reduced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of Internet, and in particular, to a data processing method and device, a readable storage medium and a program product. BACKGROUND

[0002] Tree structure data (such as file directory of computer system and product classification, etc.) is one of the core data. For example, in the file directory, the file directory usually contains multiple levels of nesting, and the file directory is a typical tree structure data. It is necessary to store the above-mentioned tree structure data, and the data structure of storage affects the query efficiency and update complexity of the tree structure data.

[0003] In some application scenarios, a user can query the above-mentioned tree structure data, for example, can query the direct child or all recursive child data of a folder through the above-mentioned tree structure data. In addition, cross-table associated tree structure (such as the relationship between folders and files) also exists universally, for example, it is necessary to query all subfolders of a certain folder and the file list of all subfolders.

[0004] In the related art, the tree structure data is queried by parent reference or materialized path. However, the parent reference method needs to be recursively queried or queried multiple times when querying all child nodes, and the query efficiency is low. Although the materialized path method can support efficient direct query or recursive query, when a node is updated, all related paths need to be updated, which is prone to errors in the updating process and affects system performance in a high concurrency scenario. SUMMARY

[0005] Embodiments of the present disclosure provide a data processing method, device, readable storage medium and program product to solve the problems of low efficiency of existing recursive child query and complex node update.

[0006] In a first aspect, the embodiments of the present disclosure provide a data processing method, comprising:

[0007] obtaining tree structure data;

[0008] constructing an in-memory data structure of a node graph based on the parent-child node association relationship in the tree structure data and the data of each node; wherein the in-memory data structure comprises a node array, a key-identity mapping table and a child node mapping table, the node array represents the association relationship between the node identity and the node data; the key-identity mapping table represents the association relationship between the key of the node and the identity of the node; and the child node mapping table represents the association relationship between the identity of the parent node and the list of child nodes corresponding to the parent node, the list of child nodes comprising the identity of at least one direct child node.

[0009] store the in-memory data structure of the node graph.

[0010] In a second aspect, the embodiments of the present disclosure provide a data processing apparatus, comprising: an obtaining unit configured to obtain tree structure data;

[0011] a constructing unit configured to construct an in-memory data structure of a node graph based on a parent-child node association relationship in the tree structure data and data of each node; wherein the in-memory data structure comprises a node array, a key-identity mapping table and a child node mapping table, the node array representing an association relationship between a node identity and node data; the key-identity mapping table representing an association relationship between a key of the node and an identity of the node; and the child node mapping table representing an association relationship between an identity of a parent node and a list of child nodes corresponding to the parent node, the list of child nodes comprising at least one identity of a direct child node.

[0012] a storing unit configured to store the in-memory data structure of the node graph.

[0013] In a third aspect, the embodiments of the present disclosure provide an electronic device, comprising: a processor and a memory;

[0014] the memory stores computer-executable instructions;

[0015] the processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method according to the first aspect and various possible designs of the first aspect.

[0016] In a fourth aspect, the embodiments of the present disclosure provide a computer-readable storage medium, the computer-readable storage medium storing computer-executable instructions, when a processor executes the computer-executable instructions, the method according to the first aspect and various possible designs of the first aspect is implemented.

[0017] In a fifth aspect, the embodiments of the present disclosure provide a computer program product, comprising a computer program, when a processor executes the computer program, the method according to the first aspect and various possible designs of the first aspect is implemented.

[0018] The data processing method, device, readable storage medium and program product provided by the embodiment, the method is used for solving the problem that the traditional scheme cannot support the recursive query and the path string analysis. The method is used for solving the problem that the traditional scheme cannot support the recursive query and the path string analysis. The method comprises the following steps: according to the parent-child node association relationship in the tree structure data and data of each node, the tree structure data is decomposed into associated triples, that is, a node array, a key-identity mapping table and a child node mapping table. The child node mapping table represents the association relationship between the identity of each parent node and the corresponding child node list of each parent node. Therefore, for the recursive query, when the key of a parent node is queried, the corresponding identity can be queried according to the key, the parent node child node list can be queried according to the identity, and the child node list query can be performed according to each child node in the child node list. The time complexity of the full sub-tree query is only O (N), where N is the size of the result set, which helps to improve the query speed. In addition, in the node graph, each node only stores its own information, and the parent-child relationship is maintained by the child node mapping table. Therefore, the update operation is localized and atomized, and does not involve the descendant nodes. When the parent node is updated, the data content of any child node does not need to be modified. The efficient movement of the entire child node list is supported. The catastrophic overhead caused by the cascading update in the materialized path scheme is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present disclosure, and other drawings can also be obtained by those skilled in the art without any creative labor.

[0020] Figure 1 A data structure diagram using a parent reference;

[0021] Figure 2 A diagram of tree structure data stored using a materialized path;

[0022] Figure 3 A flowchart of the data processing method provided by the present disclosure Figure 1 ;

[0023] Figure 4 A memory data structure diagram of a node graph;

[0024] Figure 5 A flowchart of the data processing method provided by the embodiment of the present disclosure Figure 2 ;

[0025] Figure 6 A structure block diagram of the data processing device provided by the embodiment of the present disclosure;

[0026] Figure 7A hardware structure schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0027] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some but not all of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present disclosure.

[0028] In some related technologies, a parent reference is used to store and query tree structure data. Specifically, an identifier of a parent node is directly recorded on each node to establish a relationship, and for each node, the identifier represents who is the direct parent node of the node. Figure 1 A data structure schematic diagram of a data structure using a parent reference is shown. Figure 1 As shown in the data structure, the name, folder level and parent folder of each folder are stored. For example, the parent node of "folder-1" is "root node-file directory", the parent node of "folder-1-A" is "folder-1", and so on. When querying, the direct child node of each node can be queried. However, to query all descendants of a node, recursive querying or multiple queries need to be performed.

[0029] For example, if all the child folders under "root node-file directory" are to be queried, multiple queries need to be performed: 1. Query the 2-level file directory of the parent folder "root node-folder" first. 2. After obtaining "folder-1" and "folder-2", continue to query the list of the parent folder "folder-1" and the parent folder "folder-2". Therefore, when querying all descendants of a node, the query is relatively complex and the efficiency is relatively low when using a parent reference to store and query tree structure data.

[0030] In some related technologies, a materialized path is used to store and query tree structure data. Specifically, a complete path string from a root node to a current node is recorded on each node to establish a relationship. For each node, the above path string can indicate the absolute position of the node in the tree structure. Please refer to Figure 2 , Figure 2 A data structure schematic diagram of tree structure data stored using a materialized path is shown. Figure 2 As shown in the data structure, the name, folder level and parent folder of each folder are stored. For example, the level path of "folder-1" is "root node-folder", the level path of "folder-1-A" is "root node-folder / folder-1", and so on.

[0031] When querying the parent folder "root node-folder", only the folder list including the hierarchical path prefixed with "root node-folder" needs to be queried. But if the parent folder "folder-1" is queried, two-step query is needed: 1. First, the hierarchical path with the folder name folder-1 is queried to obtain the result "root node-folder". 2. Then, the folder list prefixed with "root node-folder" is queried. When an update occurs, for example, folder-1 changes, folder-1-A and folder-1-B both need to be modified. Therefore, this scheme is not friendly to node updates.

[0032] In summary, the data structure of the tree structure data has the problems of low query efficiency when querying all descendants of a node or complex node update.

[0033] To solve the problems of low query efficiency and complex update of the data structure of the tree structure data in the related art, the scheme provided by the present disclosure converts the tree structure data into a memory structure of a node graph including a node array, a key-identifier mapping table and a child node mapping table. When querying the tree structure data, the above memory structure of the node graph can be quickly queried to reduce the query time of recursive subsets and improve the query efficiency. In addition, when updating a node, only the child node list involved needs to be modified, thus simplifying the complexity of node update.

[0034] Please refer to Figure 3 , Figure 3 the flowchart of the data processing method provided by the present disclosure Figure 1 As shown in Figure 3 , the method comprises the following steps:

[0035] S301: Obtain tree structure data.

[0036] In this embodiment, the execution subject of the data processing method can be various electronic devices, such as servers, etc., and can also be terminal devices.

[0037] Each data in the tree structure data is a node. For example, a folder can be a node, and a file can be a node.

[0038] The node graph is used in the present disclosure to describe the storage structure of the tree structure data in the memory.

[0039] S302: Based on the parent-child node relationships and the data of each node in the tree-structured data, construct an in-memory data structure for the node graph; wherein, the in-memory data structure includes a node array, a key-identifier mapping table, and a child node mapping table. The node array represents the relationship between node identity and node data; the key-identifier mapping table represents the relationship between the node key and the node identity; the child node mapping table represents the relationship between the parent node's identifier and the list of child nodes corresponding to the parent node, and the list of child nodes includes the identity of at least one direct child node.

[0040] A node's identifier serves as an index, and the node data is the value stored at that index location. The node data can describe the node's attributes. Taking a file system as an example, each node's data includes its name, type (file, directory, etc.), size, format, permissions, owner, creation time, last modification time, and latest access time.

[0041] A node's identity can be a unique identifier, such as obtained by processing node data using a preset encoding method.

[0042] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the memory data structure for a node graph. For example... Figure 4 As shown, the node graph 40 includes a node array 41, a key-identifier mapping table 42, and a child node mapping table 43. In the node array 41, the identifiers of node 1, node 2, and node 3 can be used as indexes; the data of node 1 is stored at the location indicated by the identifier of node 1, the data of node 2 is stored at the location indicated by the identifier of node 2, and the data of node 3 is stored at the location indicated by the identifier of node 3.

[0043] A key-identifier mapping table can, for example, associate the key of a storage node with its identity identifier. The key of a node can be, for example, the key corresponding to each node in the dataset to be processed. The dataset to be processed can be stored in a database as key-value pairs. Here, the key of each node can be the key of the data corresponding to that node in the database. Any node data in the above node array can be the value corresponding to the key of that node data stored in the database.

[0044] For example Figure 4 In the key-identity mapping table 42, for example, the key of node 1 and the identity of node 1, the key of node 2 and the identity of node 2, and the key of node 3 and the identity of node 3 can be associated and stored. Queries can be performed based on keys in the key-identity mapping table 42.

[0045] A child node mapping table can, for example, store each parent node and all its corresponding direct child nodes in association. (Illustrative example, such as...) Figure 4In the child node mapping table 43 in the parent node, the identity of the parent node 0 and the identity of each child node in the child node set 431 can be associated and stored. The child node set 431 can include, for example, node 1, node 2, node 3 and node 4. The identity of the parent node 1 and the identity of each child node in the child node set 432 can be associated and stored. The child node set 432 can include, for example, node 1-1 and node 1-2. The identity of the parent node 2 and the identity of each child node in the child node set 433 can be associated and stored. The child node set 433 can include, for example, node 2-1 and node 2-2.

[0046] In the isomorphic node graph, the identity of the parent node in the child node mapping table is the identity of the parent node.

[0047] In the heterogeneous node graph, the identity of the parent node in the child node mapping table is the key of the parent node.

[0048] By constructing the above-mentioned memory data structure of the node graph, the tree structure data (hierarchical relationship data set) is converted into a triple index. In the node array, each index corresponds to a unique node identity, and the node array stores the corresponding complete node data. Any node data can be directly accessed by the node identity.

[0049] In the child node mapping table, the identity of the parent node is used as the key to map to the node identity of all direct child nodes of the parent node. In the child node mapping table, breadth-first traversal or depth-first traversal can be performed, and the node identity index is used to recursively traverse the subset.

[0050] In some embodiments, step S302 includes:

[0051] Starting from the root node in the tree structure data, each node is traversed level by level, and the following operations are performed on the traversed node:

[0052] First, the node is assigned an identity, and the identity of the node and the corresponding node data are associated and stored in the node array.

[0053] Second, the identity of the node and the key of the node are associated and stored in the key-identity mapping table.

[0054] Third, in response to the existence of a parent node of the node, the identity of the node is added to the child node list corresponding to the identity of the parent node.

[0055] In one example, the identities of different nodes can be different.

[0056] Through the above initialization process of "assigning an identity, storing in an array, building a key mapping, and mounting a parent-child relationship", the original tree structure data can be converted into a lightweight, compact, and quickly traversable and updateable memory tree structure.

[0057] S303: store the memory data structure of the node graph.

[0058] In one example, the memory data structure of the node graph can be stored in the memory or the cache device for user query.

[0059] The method provided in the embodiment can decompose the tree structure data into associated triplets, i.e., a node array, a key-identity mapping table and a child node mapping table, according to the parent-child node association relationship in the tree structure data, and the child node mapping table stores the identity of each parent node and the corresponding child node list of each parent node. Therefore, for recursive query, when a key of a parent node is queried, the corresponding identity can be queried according to the key, the child node list of the parent node can be queried according to the identity, each child node in the child node list can be queried according to the child node list, and so on. The recursive query or path string analysis in the traditional scheme is avoided, and the time complexity of the full sub-tree query is only O(N), where N is the size of the result set, which helps to improve the query speed. In addition, in the node graph, each node only stores its own information, and the parent-child relationship is maintained by the child node mapping table. Therefore, the update operation is localized and atomized, and does not involve the descendant nodes. When the parent node is updated, the data content of any child node does not need to be modified. The efficient movement of the entire child node list is supported (only the parent reference of the root node needs to be changed). The catastrophic overhead caused by cascading update in the materialized path scheme is avoided.

[0060] For reference Figure 5 , Figure 5 Flowchart of the data processing method provided by the present disclosure Figure 2 As shown in Figure 5 , the method comprises the following steps:

[0061] S501: obtain tree structure data, and obtain configuration information, the configuration information comprising the type of the node graph to be generated and the configuration information of the child node mapping table matching the type;

[0062] In one example, the data storage table of the tree structure data can be obtained from the configuration file, and the configuration information can be obtained from the configuration file.

[0063] The configuration information can be used to configure the type of the node graph. The type of the node graph comprises a homogeneous node graph and a heterogeneous node graph. In the homogeneous node graph, the non-leaf node and the leaf node are of the same type, and in the heterogeneous node graph, the non-leaf node and the leaf node are of different types.

[0064] In one example, the parent node identifier field can be an identity identifier field of the parent node, such as a folder code. Illustratively, the configuration information of the child node mapping table comprises: configuring the parent node identifier field as the identity identifier field in the homogeneous node graph.

[0065] In one example, the parent node identification field can be a parent node unique identifier field (e.g., a key), such as a unique identifier generated for the parent node using a preset encoding rule. Illustratively, the configuration information of the child node mapping table includes configuring the parent node identification field as the key of the parent node in the heterogeneous node graph.

[0066] S502: Based on the parent-child node association relationship in the tree structure data and the data of each node, an in-memory data structure of the node graph is constructed; the in-memory data structure includes a node array, a key-identifier mapping table, and a child node mapping table, the node array represents the association relationship between the node identity and the node data; the key-identifier mapping table represents the association relationship between the key of the node and the identity of the node; the child node mapping table represents the association relationship between the identity of the parent node and the list of child nodes corresponding to the parent node, and the list of child nodes includes the identity of at least one direct child node.

[0067] S503: The node array, the key-identifier mapping table, and the child node list of the node graph are stored.

[0068] The process of constructing the node array and the key-identifier mapping table based on the parent-child node association relationship in the tree structure data can refer to the description of the related part of the embodiment shown in Figure 3 The description of the related part of the embodiment shown in

[0069] In some embodiments, the type includes a homogeneous node graph, and the configuration information of the child node mapping table matched with the homogeneous node graph includes that the identity of the parent node is the identity of the parent node.

[0070] For the homogeneous node graph, the business identifier field in the tree structure data can be extracted as the identity of the parent node in the child node mapping table.

[0071] In these embodiments, in the homogeneous tree (such as the folder architecture), each node usually has a globally unique and stable identity (such as a folder identifier), and these folder identifiers themselves are part of the hierarchical path and have explicit semantics.

[0072] In some embodiments, the type includes a heterogeneous node graph, and the configuration information of the child node mapping table matched with the heterogeneous node graph includes:

[0073] the associated node graph of the parent node;

[0074] the identity of the parent node is the key of the parent node.

[0075] Illustratively, the node graph of the folder employee is a heterogeneous node graph, and the parent node of the node graph is a folder. The associated node graph of the parent node can be a folder node graph. The identity of the parent node can be the key of the parent node.

[0076] In the heterogeneous node graph, the node types are diverse (such as directory, file, shortcut, tag, permission group, etc.), and different types of nodes can have the same identity (for example, the folder code "D100" and the user account "U100" can conflict in the string form, or even allow the same name in some systems). The child node mapping table uses the key (globally unique) of the parent node to avoid the confusion of parent-child relationship caused by repeated identity or mixed namespace; no matter whether the parent node is "file" or "tag", the child node list can be accurately and unambiguously indexed.

[0077] In the embodiment, the construction logic of the memory node graph is driven by the configuration information, so that the scheme can flexibly adapt to various node graph types such as homogeneous or heterogeneous, and provides structured data according to the configuration information, decoupling the data source format and the construction logic of the node graph. In addition, the configuration information explicitly indicates the type of the node graph and the identification field of the parent node, which can ensure the accuracy of the construction result of the node graph.

[0078] In Figure 3 and Figure 5 In some embodiments of the data processing method shown in the figures, the method further comprises:

[0079] In response to detecting that the tree structure data is updated, the memory data structure of the node graph is updated according to the updated data.

[0080] The tree structure data can be periodically detected for updates, and if the data is updated, the memory data structure of the node graph can be updated using the updated data.

[0081] For example, if the node data of a leaf node in the tree structure data is updated, the key of the updated node can be used to find the identity of the node through the key-identity mapping table. The identity of the node is located in the node array in the memory data structure, and the node data is replaced with the updated node data.

[0082] In some embodiments, in response to detecting that the tree structure data is updated, the memory data structure of the node graph is updated according to the updated data, comprising:

[0083] In response to detecting that the parent node of a node in the tree structure data is updated, the identity of the node is removed from the child node list of the original parent node, and the identity of the node is added to the child node list of the updated parent node.

[0084] If the parent node of a node is changed, the key of the node is used to find the identity of the node through the key-identity mapping table. The identity of the node is found in the child node mapping table through recursive query, the identity of the child node is removed from the child node list of the original parent node, and the identity of the child node is added to the child node list of the new parent node.

[0085] In these embodiments, when the parent node is updated, the update operation is localized and atomized, and does not involve the descendant nodes; the data content of any child node does not need to be modified; efficient moving of the entire child tree is supported (only the parent reference of the root node needs to be changed); and the catastrophic overhead of "cascading update" in the materialized path scheme is avoided.

[0086] In these embodiments, by updating the memory data structure of the node graph according to the tree-structured data, the memory data structure is dynamically updated along with the tree-structured data, and the accuracy of the query result can be improved.

[0087] In Figure 3 and Figure 5 Some embodiments of the data processing method shown in the specification further include the following steps:

[0088] In response to receiving the query instruction, determining the first identity corresponding to the key indicated by the query instruction from the key-identity mapping table;

[0089] Performing the following first operation:

[0090] Based on the first identity, querying the child node mapping table to determine whether there is a child node list corresponding to the first identity in the child node mapping table;

[0091] In response to determining that there is a child node list corresponding to the first identity, taking the child node list as the first query result, and recording the first query result;

[0092] Taking the identity of a child node in the child node list as a new first identity, repeatedly performing the first operation, and adding the first query result obtained by repeatedly performing the first operation to the original first query result;

[0093] Until all child nodes have no recursive child nodes, taking the last obtained first query result as the final query result.

[0094] In one example, the query is a query on an isomorphic node tree. Taking folder data as an example, a parent folder is provided, and all recursive child folders of the parent folder are queried.

[0095] First, based on the parent folder code, the identity of the parent folder is obtained by querying the key-identity mapping table.

[0096] Second, based on the identity of the parent folder, the identity of all child folders is obtained by querying the child node mapping table. The child folder list obtained in this step is added to the return result list.

[0097] Then, based on the obtained subfolder list, it is continued to be inquired whether there is a corresponding sub-list in the corresponding sub-node mapping table, if there is, it is continued to be added to the returned result list.

[0098] In one example, the query is a query to a heterogeneous node tree. A parent folder is provided, and a personnel list of all recursive subfolders of the folder is queried:

[0099] First, based on the parent folder, all recursive subfolder lists are queried based on the folder node graph.

[0100] Secondly, the recursive subfolders are traversed in turn, and then the corresponding personnel list is queried from the sub-node mapping table of the heterogeneous node tree and added to the returned result set.

[0101] Finally, the recursive query is continued until there is no recursive subfolder.

[0102] The data processing method corresponding to the above Figures 1-5 embodiment, Figure 6 is a structure block diagram of a data processing apparatus provided by the embodiments of the present disclosure. For ease of illustration, only parts related to the embodiments of the present disclosure are shown. Referring to Figure 6 , the data processing apparatus 60 comprises:

[0103] The acquisition unit 601 is configured to acquire tree structure data.

[0104] The construction unit 602 is configured to construct a memory data structure of a node graph based on the parent-child node association relationship in the tree structure data and the data of each node. The memory data structure comprises a node array, a key-identity mapping table and a sub-node mapping table. The node array represents the association relationship between the node identity and the node data. The key-identity mapping table represents the association relationship between the key of the node and the identity of the node. The sub-node mapping table represents the association relationship between the identity of the parent node and the list of sub-nodes corresponding to the parent node. The list of sub-nodes comprises at least one identity of a direct sub-node.

[0105] The storage unit 603 is configured to store the memory data structure of the node graph.

[0106] The data processing apparatus provided in the embodiments of the present disclosure can be used to: according to the parent-child node association relationship in the tree structure data, the tree structure data is disassembled into associated triplets, that is, a node array, a key-identity mapping table and a child node mapping table, and the identity of each parent node and the corresponding child node list of each parent node are stored in the child node mapping table, so that, for recursive query, when the key of a parent node is used for query, the corresponding identity can be queried according to the key, the child node list of the parent node can be queried according to the identity, and each child node in the child node list is used for child node list query, and the process is repeated. The recursive query or path string analysis in the traditional scheme is avoided, and the time complexity of the full sub-tree query is only O(N), wherein N is the size of the result set, which helps to improve the query speed. In addition, in the node graph, each node only stores its own information, and the parent-child relationship is maintained by the child node mapping table, so that the update operation is localized and atomized, and does not involve the descendant nodes; when the parent node is updated, the data content of any child node does not need to be modified; the efficient moving of the entire child node list is supported (only the parent reference of the root node needs to be changed); and the catastrophic overhead caused by cascading update in the materialized path scheme is avoided.

[0107] In an embodiment of the present disclosure, the obtaining unit 601 is further configured to:

[0108] obtain configuration information, the configuration information including a type of the node graph to be generated and configuration information of a child node mapping table matched with the type; and the constructing unit 602 is further configured to:

[0109] construct the node array and the key-identity mapping table based on the parent-child node association relationship in the tree structure data; and construct the child node mapping table based on the configuration information of the child node mapping table and the parent-child node association relationship in the tree structure data.

[0110] In an embodiment of the present disclosure, the constructing unit 602 is further configured to:

[0111] The type includes a homogeneous node graph, and the configuration information of the child node mapping table matched with the homogeneous node graph includes that the identity of the parent node is the identity of the parent node.

[0112] In an embodiment of the present disclosure,

[0113] The type includes a heterogeneous node graph, and the configuration information of the child node mapping table matched with the heterogeneous node graph includes:

[0114] the associated node graph of the parent node;

[0115] the identity of the parent node is the key of the parent node.

[0116] In an embodiment of the present disclosure, the memory data structure of the node graph is constructed based on the parent-child node association relationship in the tree structure data, including:

[0117] starting from the root node in the tree structure data, traversing each node level by level, and performing the following operations on the traversed node:

[0118] allocating an identity for the node, and storing the identity of the node and the corresponding node data in a node array in association;

[0119] storing the identity of the node and the key of the node in a key-identity mapping table in association;

[0120] in response to the node having a parent node, adding the identity of the node to the list of child nodes corresponding to the identity of the parent node.

[0121] In an embodiment of the present disclosure, the apparatus 60 further comprises an updating unit (not shown in the figure), which is configured to:

[0122] in response to detecting that the tree structure data is updated, updating the in-memory data structure of the node graph according to the updated data.

[0123] In an embodiment of the present disclosure, the updating unit is further configured to:

[0124] in response to detecting that the parent node of a node in the tree structure data is updated, removing the identity of the node from the list of child nodes of the original parent node, and adding the identity of the node to the list of child nodes of the updated parent node.

[0125] To implement the above-mentioned embodiments, the present disclosure further provides an electronic device.

[0126] Reference Figure 7 which shows a structural schematic diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure, which can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDA), tablet computers, portable multimedia players (PMP), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 7 The electronic device shown is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0127] As Figure 7As shown, the electronic device 700 can include a processing device (e.g., a central processor, a graphics processor, etc.) 701 that can perform various suitable actions and processes in accordance with programs stored in a Read Only Memory (ROM) 702 or loaded from a storage device 708 into a Random Access Memory (RAM) 703. Various programs and data required by the electronic device 700 for operation are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other by a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0128] Generally, the following devices can be connected to the I / O interface 705: input devices 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, etc.; storage devices 708 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 709. The communication devices 709 can allow the electronic device 700 to communicate wirelessly or wired with other devices to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that not all of the shown devices are required to be implemented or present. More or fewer devices can alternatively be implemented or present.

[0129] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carrying out the methods shown in the flowcharts, the computer program being embodied on a computer readable storage medium. In such embodiments, the computer program can be downloaded and installed from a network through the communication devices 709, or installed from the storage devices 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the methods of embodiments of the present disclosure are performed.

[0130] It should be noted that the computer-readable storage medium described above in the present disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. The computer-readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal that propagates in a baseband or as part of a carrier wave, carrying computer-readable program code. Such a propagated data signal can take many forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium that can send, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to wire, cable, optical fiber, RF (radio frequency), or any suitable combination thereof.

[0131] The computer-readable storage medium described above can be included in the electronic device described above; or can exist separately from the electronic device and be not assembled in the electronic device.

[0132] The computer-readable storage medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the embodiments described above.

[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0135] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0136] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0137] The electronic device, the computer readable storage medium, and the computer program product provided in the embodiments of the present disclosure can decompose tree structure data into associated triples, i.e., a node array, a key-identity mapping table, and a child node mapping table, by associating parent-child nodes in the tree structure data, and store the identity of each parent node and the corresponding child node list in the child node mapping table, so that, for recursive queries, when a key of a parent node is queried, the corresponding identity can be queried according to the key, the child node list of the parent node can be queried according to the identity, each child node in the child node list can be queried according to the child node list, and so on. The recursive query or path string analysis in the traditional scheme is avoided, and the time complexity of the full sub-tree query is only O(N), where N is the size of the result set, which helps to improve the query speed. In addition, in the node graph, each node only stores its own information, and the parent-child relationship is maintained by the child node mapping table, so that the update operation is localized and atomized, and does not involve the descendant nodes; when the parent node is updated, the data content of any child node does not need to be modified; efficient moving of the whole sub-tree is supported (only the parent reference of the root node needs to be changed); and the catastrophic overhead of "cascading update" in the materialized path scheme is avoided.

[0138] In a first aspect, according to one or more embodiments of the present disclosure, a data processing method is provided, comprising:

[0139] obtaining tree structure data;

[0140] constructing an in-memory data structure of a node graph based on the parent-child node association relationship in the tree structure data and the data of each node; wherein the in-memory data structure comprises a node array, a key-identity mapping table, and a child node mapping table, the node array represents the association relationship between the node identity and the node data; the key-identity mapping table represents the association relationship between the key of the node and the identity of the node; and the child node mapping table represents the association relationship between the identity of the parent node and the child node list corresponding to the parent node, and the child node list comprises the identity of at least one direct child node;

[0141] storing the in-memory data structure of the node graph.

[0142] According to one or more embodiments of the present disclosure, the method further comprises:

[0143] obtaining configuration information, the configuration information comprising the type of the node graph to be generated and the configuration information of the child node mapping table matching the type; and constructing the in-memory data structure of the node graph based on the parent-child node association relationship in the tree structure data, comprising:

[0144] constructing the node array and the key-identity mapping table based on the parent-child node association relationship in the tree structure data; and constructing the child node mapping table based on the configuration information of the child node mapping table and the parent-child node association relationship in the tree structure data.

[0145] According to one or more embodiments of the present disclosure, the type includes a homogeneous node graph, and the configuration information of the child node mapping table matched with the homogeneous node graph includes: an identity of the parent node as an identity of the parent node.

[0146] According to one or more embodiments of the present disclosure, the type includes a heterogeneous node graph, and the configuration information of the child node mapping table matched with the heterogeneous node graph includes:

[0147] An associated node graph of the parent node.

[0148] An identity of the parent node as a key of the parent node.

[0149] According to one or more embodiments of the present disclosure, based on the parent-child node association relationship in the tree structure data, a memory data structure of a node graph is constructed, including:

[0150] Starting from a root node in the tree structure data, each node is traversed level by level, and the following operations are performed on the traversed node:

[0151] An identity of the node is allocated, and the identity of the node and the corresponding node data are associated and stored in a node array;

[0152] The identity of the node and the key of the node are associated and stored in a key-identity mapping table;

[0153] In response to the node having a parent node, the identity of the node is added to a child node list corresponding to the identity of the parent node.

[0154] According to one or more embodiments of the present disclosure, the method further includes:

[0155] In response to detecting that the tree structure data is updated, updating the memory data structure of the node graph according to the updated data.

[0156] According to one or more embodiments of the present disclosure, in response to detecting that the tree structure data is updated, updating the memory data structure of the node graph according to the updated data, including:

[0157] In response to detecting that a parent node of a node in the tree structure data is updated, the identity of the node is removed from a child node list of the original parent node, and the identity of the node is added to a child node list of the updated parent node.

[0158] In a second aspect, according to one or more embodiments of the present disclosure, a data processing apparatus is provided, including:

[0159] An acquisition unit, configured to acquire tree structure data.

[0160] The constructing unit is configured to construct an in-memory data structure of the node graph based on the parent-child node association relationship in the tree structure data and the data of each node. The in-memory data structure includes a node array, a key-identity mapping table, and a child node mapping table. The node array represents the association relationship between the node identity and the node data. The key-identity mapping table represents the association relationship between the key of the node and the identity of the node. The child node mapping table represents the association relationship between the identity of the parent node and the child node list corresponding to the parent node. The child node list includes the identity of at least one direct child node.

[0161] The storage unit is configured to store the in-memory data structure of the node graph.

[0162] According to one or more embodiments of the present disclosure, the obtaining unit is further configured to:

[0163] obtain configuration information, the configuration information including the type of the node graph to be generated and configuration information of the child node mapping table matched with the type; and the constructing unit is further configured to:

[0164] construct the node array and the key-identity mapping table based on the parent-child node association relationship in the tree structure data; and construct the child node mapping table based on the configuration information of the child node mapping table and the parent-child node association relationship in the tree structure data.

[0165] According to one or more embodiments of the present disclosure, the constructing unit is further configured to:

[0166] The type includes a homogeneous node graph, and the configuration information of the child node mapping table matched with the homogeneous node graph includes that the identity of the parent node is the identity of the parent node.

[0167] According to one or more embodiments of the present disclosure, the type includes a heterogeneous node graph, and the configuration information of the child node mapping table matched with the heterogeneous node graph includes:

[0168] the associated node graph of the parent node;

[0169] the identity of the parent node is the key of the parent node.

[0170] According to one or more embodiments of the present disclosure, the constructing unit is further configured to:

[0171] starting from the root node in the tree structure data, traversing each node level by level, and performing the following operations on the traversed node:

[0172] allocating an identity to the node, and storing the identity of the node and the corresponding node data in the node array in association;

[0173] storing the identity of the node and the key of the node in the key-identity mapping table in association.

[0174] In response to the node having a parent node, adding the identity of the node to a child node list corresponding to the identity of the parent node.

[0175] According to one or more embodiments of the present disclosure, the apparatus further includes an updating unit configured to:

[0176] In response to detecting that the tree structure data is updated, updating the in-memory data structure of the node graph according to the updated data.

[0177] According to one or more embodiments of the present disclosure, the updating unit is further configured to:

[0178] In response to detecting that a parent node of a node in the tree structure data is updated, removing the identity of the node from a child node list of the original parent node and adding the identity of the node to a child node list of the updated parent node.

[0179] In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including at least one processor and a memory;

[0180] The memory stores computer-executable instructions;

[0181] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method of the first aspect above and various possible designs of the first aspect.

[0182] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, and the computer-readable storage medium stores computer-executable instructions, when a processor executes the computer-executable instructions, the method of the first aspect above and various possible designs of the first aspect are implemented.

[0183] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, and when a processor executes the computer program, the method of the first aspect above and various possible designs of the first aspect are implemented.

[0184] The above description is merely preferred embodiments of the present disclosure and a description of principles of applied technologies. It should be understood by those skilled in the art that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combinations of the above technical features or equivalent features without departing from the disclosed concept. For example, the above technical features can be replaced with technical features disclosed in the present disclosure (but not limited to) having similar functions to form technical solutions.

[0185] Moreover, while operations are depicted in a particular order, this should not be understood as requiring such an order nor infringing on the scope of the disclosure. Certain of the operations described in the discussion are combinable into a single operation, and certain operations can be separated into several operations. In some embodiments, the operations described in the discussion can be performed in an order different than presented in the discussion. In some embodiments, the operations described in the discussion can be performed concurrently. Also, while several specific implementation details are discussed in the discussion, these should not be interpreted as limiting the scope of the disclosure. Rather, certain features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0186] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A data processing method, characterized in that, include: Obtain tree-structured data; Based on the parent-child node relationships and the data of each node in the tree-structured data, an in-memory data structure for the node graph is constructed. This in-memory data structure includes a node array, a key-identifier mapping table, and a child node mapping table. The node array represents the relationship between node identifiers and node data. The key-identifier mapping table represents the relationship between the key of a node and the identifier of that node. The child node mapping table represents the relationship between the identifier of a parent node and the list of child nodes corresponding to that parent node, where the list of child nodes includes the identifier of at least one direct child node. A memory data structure for storing the node graph.

2. The method according to claim 1, characterized in that, The method further includes: Obtain configuration information, including the type of the node graph to be generated and the configuration information of the child node mapping table matching the type; and construct the memory data structure of the node graph based on the parent-child node association relationship in the tree structure data, including: Based on the parent-child node relationships in the tree structure data, construct the node array and the key-identifier mapping table; The child node mapping table is constructed based on the configuration information of the child node mapping table and the parent-child node association relationship in the tree structure data.

3. The method according to claim 2, characterized in that, The type includes a homogeneous node graph, and the configuration information of the child node mapping table matching the homogeneous node graph includes: the identifier of the parent node is the identity identifier of the parent node.

4. The method according to claim 2, characterized in that, The type includes heterogeneous node graphs, and the configuration information of the child node mapping table matching the heterogeneous node graph includes: The node graph of the parent node; The identifier of the parent node is the key of the parent node.

5. The method according to claim 1, characterized in that, The in-memory data structure for constructing a node graph based on the parent-child node relationships and the data of each node in the tree structure data includes: Starting from the root node in the tree structure data, traverse each node level by level, and perform the following operations on the traversed nodes: Assign an identity identifier to the node, and associate the node's identity identifier with the corresponding node data and store them in a node array; The identity identifier of the node is associated with the key of the node and stored in a key-identifier mapping table; In response to the existence of a parent node, the identity identifier of the node is added to the list of child nodes corresponding to the identifier of the parent node.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: In response to the detection of an update to the tree structure data, the memory data structure of the node graph is updated according to the updated data.

7. The method according to claim 6, characterized in that, The step of updating the memory data structure of the node graph according to the updated data in response to detecting an update in the tree structure data includes: In response to detecting an update of the parent node of a node in the tree structure data, the identity of the node is removed from the list of child nodes of the original parent node, and the identity of the node is added to the list of child nodes of the updated parent node.

8. The method according to any one of claims 1-5, characterized in that, The method further includes: In response to receiving a query command, determine the first identity identifier corresponding to the key indicated by the query command from the key-identity mapping table; Perform the following first operation: Based on the first identity identifier, query the sub-node mapping table to determine whether there is a list of sub-nodes in the sub-node mapping table that corresponds to the first identity identifier; In response to determining that there is a list of child nodes corresponding to the first identity identifier, the list of child nodes is used as the first query result, and the first query result is recorded; Take the identity identifier of a child node in the child node list as the new first identity identifier, repeat the first operation, and add the first query result obtained by repeating the first operation to the original first query result; Continue until all child nodes have no recursive children, then use the first query result obtained last as the final query result.

9. A data processing apparatus, characterized in that, include: The acquisition unit is used to acquire tree-structured data. A construction unit is used to construct an in-memory data structure of a node graph based on the parent-child node relationships and the data of each node in the tree-structured data. The in-memory data structure includes a node array, a key-identifier mapping table, and a child node mapping table. The node array represents the relationship between node identifiers and node data. The key-identifier mapping table represents the relationship between the key of a node and the identifier of that node. The child node mapping table represents the relationship between the identifier of a parent node and the list of child nodes corresponding to that parent node, wherein the list of child nodes includes the identifier of at least one direct child node. A storage unit is used to store the memory data structure of the node graph.

10. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images