A method, device and equipment for constructing a dictionary tree and a storage medium

Abstract

The application discloses a method and device for constructing a dictionary tree, an apparatus and a storage medium. The method comprises the following steps: obtaining an external identifier of point data in graph data and an internal identifier of the point data; and constructing a dictionary tree according to the external identifier of the point data and the internal identifier of the point data. According to the technical scheme of the application, the mapping relationship between the internal identifier and the external identifier of the point data can be quickly obtained based on the constructed dictionary tree, so that the problems of low efficiency, long time consumption and large disk or memory consumption during the process of storing the graph data into a database or a memory are solved, and the performance during data import is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, and particularly relate to a method and device for constructing a dictionary tree, an apparatus and a storage medium. BACKGROUND

[0002] The data stored in the graph database is a collection of data of points and edges in the graph. The points and edges can also have their own attribute data. In the storage process of our distributed graph database system, each point data needs to be mapped to a unique internal identifier maintained internally by the graph database (i.e., the mapping of the external identifier of the point data to the internal identifier) during storage in the graph database. Each edge data needs to obtain the internal identifiers of its two endpoints to convert the edge data into data that can be stored in the graph database according to the external identifiers. The efficiency of obtaining the mapping relationship between the internal identifier and the external identifier of the point data will greatly affect the data loading capacity of our graph database. Therefore, how to quickly obtain the mapping relationship between the internal identifier and the external identifier of the point data has become a problem to be solved. SUMMARY

[0003] Embodiments of the present application provide a method and device for constructing a dictionary tree, an apparatus and a storage medium, which can quickly obtain the mapping relationship between the internal identifier and the external identifier of the point data based on the constructed dictionary tree, thereby solving the problems of low efficiency, long time and large disk or memory consumption when graph data is imported into the database or memory, and further improving the performance during data import.

[0004] According to an aspect of the present application, a method for constructing a dictionary tree is provided, comprising:

[0005] obtaining the external identifier of the point data and the internal identifier of the point data in the graph data;

[0006] constructing a dictionary tree according to the external identifier of the point data and the internal identifier of the point data.

[0007] According to another aspect of the present application, a device for constructing a dictionary tree is provided, comprising:

[0008] an obtaining module configured to obtain the external identifier of the point data and the internal identifier of the point data in the graph data;

[0009] a constructing module configured to construct a dictionary tree according to the external identifier of the point data and the internal identifier of the point data.

[0010] According to another aspect of the present application, an electronic device is provided, comprising:

[0011] at least one processor; and

[0012] a memory connected to the at least one processor in communication; wherein,

[0013] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the dictionary tree construction method according to any one of the embodiments of the present application.

[0014] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the dictionary tree construction method according to any one of the embodiments of the present application when executed by the processor.

[0015] The embodiments of the present application can obtain the external identifier of the point data and the internal identifier of the point data in the graph data, and construct a dictionary tree according to the external identifier of the point data and the internal identifier of the point data, so as to quickly obtain the mapping relationship between the internal identifier and the external identifier of the point data based on the constructed dictionary tree, thereby solving the problems of low efficiency, long time consumption and large disk or memory consumption when the graph data is imported into a database or a memory, and further improving the performance during data import.

[0016] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced as follows. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be considered as limiting the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0018] Figure 1 is a flow chart of a dictionary tree construction method in the embodiments of the present application;

[0019] Figure 2 is a storage flow chart of point data in the embodiments of the present application;

[0020] Figure 3 is a storage flow chart of edge data in the embodiments of the present application;

[0021] Figure 4 is a structure diagram of a dictionary tree in the embodiments of the present application;

[0022] Figure 5 is a diagram of an encoded dictionary tree in the embodiments of the present application;

[0023] Figure 6is a structural schematic diagram of a dictionary tree construction device in an embodiment of the present application.

[0024] Figure 7 is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work should fall within the protection scope of the present application.

[0026] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment one

[0028] Figure 1 A flowchart of a dictionary tree construction method provided in an embodiment of the present application, the present embodiment can be applicable to the case of dictionary tree construction, the method can be executed by the dictionary tree construction device in the embodiment of the present application, and the device can be realized in the form of software and / or hardware, as shown in the figure, the method specifically includes the following steps: Figure 1

[0029] S110, obtaining the external identifier of the point data and the internal identifier of the point data in the graph data.

[0030] The external identifier of the point data can be the UID of the point data, and the UID of the point data is determined according to the source of the point data.

[0031] ​The internal identifier of the point data is an identifier allocated by the system for each point data, and a specific allocation manner can be that: the system allocates a new count as the internal identifier of the point data for each received point data, and after allocating a new count, the count is increased by 1, when receiving the next point data, the system allocates the count increased by 1 to the point data as the internal identifier of the next point data, for example, receiving point data A, the system allocates count 1, the internal identifier of point data A is 1, receiving point data B, the system allocates count 2, the internal identifier of point data B is 2, receiving point data C, the system allocates count 3, the internal identifier of point data C is 3, and so on.

[0032] It should be noted that if a repeated point is received, the internal identifier of the repeated point will not be newly allocated, wherein the repeated point is the same point data as the historically received point data, and the specific determination manner of the repeated point is that in the process of constructing the dictionary tree, it is found that the internal identifier corresponding to the ending character of the point data has been updated, and then it is determined that the point data is a repeated point.

[0033] S120, constructing a dictionary tree according to the external identifier of the point data and the internal identifier of the point data.

[0034] Specifically, the manner of constructing the dictionary tree according to the external identifier of the point data and the internal identifier of the point data can be that: constructing a node of the dictionary tree according to each character of the external identifier of the point data, position information of each character and the internal identifier of the point data; constructing an edge of the dictionary tree according to each character of the external identifier of the point data and the position relationship between the characters; and constructing the dictionary tree according to the node of the dictionary tree and the edge of the dictionary tree. The manner of constructing the dictionary tree according to the external identifier of the point data and the internal identifier of the point data can also be that: constructing a node of the dictionary tree according to each character of the external identifier of the point data and position information of each character; constructing an edge of the dictionary tree according to each character of the external identifier of the point data and the position relationship between the characters; and constructing the dictionary tree according to the node of the dictionary tree and the edge of the dictionary tree. The manner of constructing the dictionary tree according to the external identifier of the point data and the internal identifier of the point data can also be that: constructing a node of the dictionary tree according to each character of the external identifier of the point data, position information of each character and a first numerical value; constructing an edge of the dictionary tree according to each character of the external identifier of the point data and the position relationship between the characters; and constructing the dictionary tree according to the node of the dictionary tree and the edge of the dictionary tree. The embodiments of the present application do not limit this.

[0035] Optionally, constructing the dictionary tree according to the external identifier of the point data and the internal identifier of the point data comprises:

[0036] constructing a node of the dictionary tree according to each character of the external identifier of the point data, position information of each character and the internal identifier of the point data;

[0037] constructing edges of the dictionary tree according to each character of the external identifier of the point data and a position relationship between the characters;

[0038] constructing the dictionary tree according to the nodes of the dictionary tree and the edges of the dictionary tree.

[0039] The position information of the character can be a non-end character or an end character. For example, if the external identifier is ABLE, the position information of the characters A, B and L are non-end characters, and the position information of the character E is an end character.

[0040] The position relationship between the characters is an order of the characters. For example, if the external identifier is ABLE, the position relationship between the characters is that A is before B, B is before L, and L is before E, or the position relationship between the characters is that B is after A, L is after B, and E is after L.

[0041] Specifically, the manner of constructing the nodes of the dictionary tree according to each character of the external identifier of the point data, the position information of each character and the internal identifier of the point data can be that determining the internal identifier corresponding to the node of the Nth character according to the Nth character of the external identifier of the point data, the position information of the Nth character and the internal identifier of the point data, where N is a positive integer greater than or equal to 1; and constructing the nodes of the dictionary tree according to the internal identifier corresponding to each node.

[0042] Specifically, the manner of constructing the edges of the dictionary tree according to each character of the external identifier of the point data and the position relationship between the characters can be that constructing the edges of the dictionary tree according to the order of the characters of the external identifier. For example, if the external identifier is ABLE, the position relationship between the characters is that B is after A, L is after B, and E is after L, A is an edge connecting a first node and a second node, B is an edge connecting the second node and a third node, L is an edge connecting the third node and a fourth node, and E is an edge connecting the fourth node and a fifth node.

[0043] Optionally, the constructing the nodes of the dictionary tree according to each character of the external identifier of the point data, the position information of each character and the internal identifier of the point data comprises:

[0044] determining the internal identifier corresponding to the node of the Nth character according to the Nth character of the external identifier of the point data, the position information of the Nth character and the internal identifier of the point data, where N is a positive integer greater than or equal to 1;

[0045] constructing the nodes of the dictionary tree according to the internal identifier corresponding to each node.

[0046] The Nth character corresponding node can be a child node with the Nth character as an edge, for example, if the external identifier is ABLE, the parent node with A as an edge is a root node, and the internal identifier of the child node with A as an edge is determined according to the position information of the character A.

[0047] Specifically, the way of determining the internal identifier of the Nth character corresponding node according to the Nth character of the external identifier of the point data, the position information of the Nth character, and the internal identifier of the point data can be: determining the Nth character of the external identifier as a target character, if the target character is not a last character, determining a first value as the internal identifier of the Nth character corresponding node; if the target character is a last character of the target external identifier, determining the internal identifier corresponding to the target external identifier as the internal identifier of the Nth character corresponding node.

[0048] Specifically, the way of constructing the nodes of the dictionary tree according to the internal identifier corresponding to each node can be: constructing the nodes of the dictionary tree according to the internal identifier corresponding to each node and the node attribute information of each node, wherein the node attribute information includes: leaf node or non-leaf node, for example, if the external identifier is ABLE, the parent node with A as an edge is a root node, and the character A is not a last character of ABLE, it is determined that the child node with A as an edge is a non-leaf node, and the internal identifier of the child node with A as an edge is a first value.

[0049] Optionally, determining the internal identifier corresponding to the child node of the Nth node according to the Nth character of the external identifier of the point data, the position information of the Nth character, and the internal identifier of the point data includes:

[0050] If it is determined according to the position information of the Nth character that the Nth character is not a last character, the internal identifier of the Nth character corresponding node is determined as a first value;

[0051] If it is determined according to the position information of the Nth character that the Nth character is a last character of the target external identifier, the internal identifier of the Nth character corresponding node is determined as the internal identifier corresponding to the target external identifier.

[0052] The first value is a negative value, and it should be noted that the internal identifier is a non-negative value when the system assigns the internal identifier, and the first value is selected to be negative to distinguish from the internal identifier.

[0053] It should be noted that the dictionary tree can be constructed in the following manner: each time a tree structure is constructed according to a preset number of point data external identifiers, until all point data external identifiers are traversed, and finally a dictionary tree is obtained, wherein the preset number can be a positive integer greater than or equal to 1. For example, a sub-tree structure can be constructed according to the external identifier ABLE, the parent node with A as the edge is the root node, character A is not the last character of ABLE, it is determined that the child node with A as the edge is a non-leaf node, and the internal identifier of the child node with A as the edge is the first value; the parent node with B as the edge is the child node with A as the edge, character B is not the last character of ABLE, it is determined that the child node with B as the edge is a non-leaf node, and the internal identifier of the child node with B as the edge is the first value; the parent node with L as the edge is the child node with B as the edge, character L is not the last character of ABLE, it is determined that the child node with L as the edge is a non-leaf node, and the internal identifier of the child node with L as the edge is the first value; the parent node with E as the edge is the child node with L as the edge, character E is the last character of ABLE, it is determined that the child node with E as the edge is a leaf node, and the internal identifier of the child node with E as the edge is the internal identifier of ABLE. The above-constructed sub-tree structure is updated according to the external identifier AS. Since A is not the last character of the external identifier AS, the child node with A as the edge remains unchanged, the parent node with S as the edge is the child node with A as the edge, and no new node needs to be added. It is queried whether the current sub-tree structure includes the child node with S as the edge. If it exists, the child node with S as the edge does not need to be added. If it does not exist, the child node with S as the edge is added, character S is the last character of AS, it is determined that the child node with S as the edge is a leaf node, and the internal identifier of the child node with S as the edge is the internal identifier of AS. If the sub-tree structure already has a child node with S as the edge, and the internal identifier of the child node with S as the edge is the first value, the internal identifier of the child node with S as the edge is updated to the internal identifier of AS. Similarly, this will not be described in detail here.

[0054] Optionally, it further comprises:

[0055] Obtaining attribute information of point data in the graph data, external identifier of the starting point of edge data, external identifier of the ending point of edge data, and attribute information of the edge data;

[0056] Generating a point data file according to the external identifier of the point data, the internal identifier of the point data, and the attribute information of the point data, and storing it to the disk;

[0057] According to the external identifier of the starting point of edge data and the external identifier of the ending point of edge data, the dictionary tree is queried to obtain the internal identifier of the starting point of edge data and the internal identifier of the ending point of edge data;

[0058] Generate the edge data file according to the internal identifier of the start point of the edge data, the internal identifier of the end point of the edge data, and the attribute information of the edge data, and store it to the disk.

[0059] Specifically, the manner of generating the point data file according to the external identifier of the point data, the internal identifier of the point data, and the attribute information of the point data and storing it to the disk can be that the point data file is generated according to the external identifier of the point data, the internal identifier of the point data, and the attribute information of the point data, and the point data file is stored to the disk, and part of the point data files is filtered from the point data file according to a preset rule, and the part of the point data files filtered is stored to the memory. For example, the key file in the point data file can be stored to the memory, so that when the value file is accessed through the key file, the key file does not need to be taken out from the disk first, and then the value file is read according to the key file, and the efficiency of accessing the value file is improved.

[0060] Specifically, the manner of generating the edge data file according to the internal identifier of the start point of the edge data, the internal identifier of the end point of the edge data, and the attribute information of the edge data and storing it to the disk can be that the edge data file is generated according to the internal identifier of the start point of the edge data, the internal identifier of the end point of the edge data, and the attribute information of the edge data, and the edge data file is stored to the disk, and part of the edge data files is filtered and stored to the memory. For example, part of the files in the edge data file can be stored to the memory, so that the edge data file is read, and the efficiency of accessing the edge data file is improved.

[0061] It should be noted that when the point data file and the edge data file are stored to the disk, the point data file and the edge data file can be first split to obtain at least two data files, and then the files obtained after the splitting are stored to the disk.

[0062] Optionally, generating the point data file according to the external identifier of the point data, the internal identifier of the point data, and the attribute information of the point data and storing it to the disk comprises:

[0063] Determining a key value of the point data according to the external identifier of the point data and the internal identifier of the point data;

[0064] Determining a value of the point data according to the external identifier of the point data and the attribute information of the point data;

[0065] Generating a key file according to the key value of the point data;

[0066] Generating a value file according to the value of the point data;

[0067] Store the key file and value file to the disk.

[0068] The key is a pair of (UID, innerID) and the value is a pair of (innerID, attribute value of point data). UID is the external identifier of point data and innerID is the internal identifier of point data.

[0069] like Figure 2 As shown, Figure 2 A flowchart for importing point data from a graph. Figure 2 In the code, Prop1 contains the attribute information of the point data corresponding to UID1, and Prop2 contains the attribute information of the point data corresponding to UID2. `idTrie` is a trie. The innerID of point data 1 is 1, and the innerID of point data 2 is 2. A key is generated based on UID1 and 1 of point data 1, and a key is generated based on UID2 and 2 of point data 2. A value is generated based on the attribute information Prop1 and 1 of point data 1, and a value is generated based on the attribute information Prop2 and 2 of point data 2. A key file is obtained based on the generated key values, and a value file is obtained based on the generated value values. The key and value files are stored on disk, and a portion of the key files are cached in memory.

[0070] It should be noted that the UID of the point data can be used to directly construct an idTrie, or it can be stored on disk first and then the idTrie can be constructed on demand, thus avoiding memory waste. Several key or value records are combined into key files or value files, which are then written to disk. (Some) key files can be cached in memory.

[0071] like Figure 3 As shown, Figure 3The storage process flowchart of the edge data, srcUID1 is the external identifier of the starting point of the edge data 1, dstUID1 is the external identifier of the terminal point of the edge data 1, srcUID2 is the external identifier of the starting point of the edge data 2, and dstUID2 is the external identifier of the terminal point of the edge data 2. The attribute information of the edge data 1 is Prop1, and the attribute information of the edge data 2 is Prop2. According to the srcUID1, the idTrie is queried, the innerID corresponding to the srcUID1 is 1, the innerID corresponding to the dstUID1 is 2, the innerID corresponding to the srcUID2 is 5, and the innerID corresponding to the dstUID2 is 6. According to the innerID corresponding to the srcUID1, the innerID corresponding to the dstUID1, the innerID corresponding to the srcUID2, the innerID corresponding to the dstUID2, Prop1 and Prop2, the edge data file is generated, the edge data file is stored to the disk, and part of the edge data file is cached to the memory.

[0072] It should be noted that the idTrie can be generated when the point data is imported. However, if the idTrie fails to be generated in the previous process, the key file of the corresponding point can be pulled from the disk to quickly construct the idTrie. By querying the idTrie, the original edge data is mapped into the internal edge data stored in the graph database, and after the edge data file is formed, it is stored in the disk. At the same time, the (part of) edge data can also be written into the memory as a cache.

[0073] Optionally, the dictionary tree is queried according to the external identifier of the starting point of the edge data and the external identifier of the terminal point of the edge data, and the internal identifier of the starting point of the edge data and the internal identifier of the terminal point of the edge data are obtained.

[0074] The dictionary tree is encoded to obtain target data structure information.

[0075] The target data structure information is queried according to the external identifier of the starting point of the edge data and the external identifier of the terminal point of the edge data, and the internal identifier of the starting point of the edge data and the internal identifier of the terminal point of the edge data are obtained.

[0076] The target data structure information can include node attribute information and internal identifier corresponding to each character. The target data structure information can also include node attribute information, internal identifier of the node and next character corresponding to each character. It should be noted that if the character is a terminal character, the target data structure information includes a leaf node and the internal identifier of the node. If the character is not a terminal character, the target data structure information includes a non-leaf node, no internal identifier, a next character and storage location information of the next character in the memory.

[0077] Optionally, the dictionary tree is encoded to obtain target data structure information, including:

[0078] The pop order of the externally identified characters is determined according to the dictionary tree;

[0079] The write position information of each externally identified character is determined according to the pop order of the externally identified characters;

[0080] The node attribute information, the internal identifier of the node and the next character corresponding to each character are determined according to the dictionary tree;

[0081] The target data structure information is determined according to the write position information of each character, the node attribute information corresponding to each character, the internal identifier of the node and the next character.

[0082] It should be noted that the dictionary tree is constructed from right to left.

[0083] In a specific example, as shown in Figure 4 There are always four point data: (ABLE, 1), (APP, 2), (APPLE, 3), (AS, 4), the tree structure starts from a root node, and the mapping structure is constructed starting from the public starting letter A of the four UIDs. Since A is not the end character of the UID, the node has no corresponding innerID (so it is set to -1). The letters in the second position of each UID are B, P and S, respectively, so three branches are generated here. Taking AS as an example, S is the last letter of the UID AS, so the innerID is set to 4 at this node position. In this way, when all UIDs are traversed, the complete structure is constructed. The specific structure of the dictionary tree is shown in Figure 4 The parent node with A as the edge is the root node, the child node with A as the edge is a non-leaf node (Leaf = false), and the memory identifier is -1. The parent node with S as the edge is the child node with A as the edge, and the child node with S as the edge is a leaf node (Leaf = true), and the memory identifier is 4 (the memory identifier corresponding to AS is 4). The remaining nodes of the dictionary tree are not described here. It should be noted that when searching for a certain UID, all UIDs need to be traversed, and the corresponding path in the dictionary tree is found to obtain the corresponding innerID, and the time complexity is O(length(UID)). Figure 4

[0084] If the objects on the heap are used to describe such a tree structure, memory fragmentation is easy to occur, increasing the garbage collection pressure of the Java virtual machine. Therefore, the encoded is selected to store this data structure, as shown in Figure 5 ​As shown, a stack is used to store nodes still in the process of building (whose child nodes are not yet all determined). When a node is popped from the stack, it can be encoded into contiguous memory. As illustrated, after adding "ABLE", the stack sequentially adds the nodes "A", "B", "L", and "E". When "APP" is added, "B", "L", and "E" are popped from the stack in reverse order and written to memory positions 0, 1, and 2 in the order they were popped. When "APPLE" is added, only an additional node is added to the original "APP" path; no node is popped from the stack. When "AS" is added, "P", "P", "L", and "E" are popped from the stack in reverse order and written to positions 3-6. This process continues until the trie is fully constructed. All nodes are popped from the stack, and the character "E" is written to memory position 0, with the corresponding node... The dot is a leaf node with a memory identifier of 1. The character L is written in memory position 1. The node corresponding to the character L is not a leaf node and has no memory identifier. The next character is the character E, which is written in memory position 0. The character B is written in memory position 2. The node corresponding to the character B is not a leaf node and has no memory identifier. The next character is the character L, which is written in memory position 1. The character E is written in memory position 3. The node corresponding to the character E is a leaf node with a memory identifier of 2. The character L is written in memory position 4. The node corresponding to the character L is not a leaf node and has no memory identifier. The next character is the character E. The character 'E' is written in memory location 3, the character 'P' is written in memory location 5, the node corresponding to the character 'P' is not a leaf node, and its memory identifier is 2. The next character is the character 'L', written in memory location 4. The character 'P' is written in memory location 6, the node corresponding to the character 'P' is not a leaf node, and it has no memory identifier. The next character is the character 'P', written in memory location 5. The character 'S' is written in memory location 7, the node corresponding to the character 'S' is a leaf node, and its memory identifier is 4. The character 'A' is written in memory location 8, the node corresponding to the character 'A' is not a leaf node, and it has no memory identifier. The next character is written in memory location 8. The character 'B' is written at memory position 2, the character 'P' at memory position 6, and the character 'S' at memory position 7. The root node is written at memory position 9. The root node is not a leaf node, has no memory identifier, and the next character is the character 'A' written at memory position 8. Through the above steps, memory encoding is completed (address encoding can also automatically select the absolute address or address offset, saving memory space). During the construction process, the total number of active nodes in the stack never exceeds length(UID)*length(charset), where charset is all the non-repeating characters appearing in UID. The memory structure after encoding can achieve the same lookup algorithm as a trie through address jumps, but its memory utilization efficiency is far superior to that of a standard trie implementation.

[0085] In this embodiment of the invention, the data file is stored at the bottom layer of the LSM Tree storage structure, which avoids the situation of overlapping data files and eliminates the need to compare whether data files overlap, thus improving system performance.

[0086] The technical solution of this embodiment obtains the external identifier and internal identifier of the point data in the graph data; and constructs a trie based on the external identifier and internal identifier of the point data. Based on the constructed trie, the mapping relationship between the internal identifier and external identifier of the point data can be quickly obtained, thereby solving the problems of low efficiency, long time and large disk or memory consumption when graph data is put into the database or memory, and thus improving the performance of data import.

[0087] Example 2

[0088] Figure 6 This is a schematic diagram of a trie construction device provided in an embodiment of the present invention. This embodiment is applicable to trie construction, and the device can be implemented using software and / or hardware. The device can be integrated into any device that provides trie construction functionality, such as… Figure 6 As shown, the trie construction device specifically includes: an acquisition module 610 and a construction module 620.

[0089] The acquisition module is used to acquire the external identifier and internal identifier of the point data in the graph data.

[0090] A construction module is used to construct a trie based on the external identifier and the internal identifier of the point data.

[0091] The above-described products can perform the methods provided in any embodiment of the present invention, and have the corresponding functional modules and beneficial effects for performing the methods.

[0092] The technical solution of this embodiment obtains the external identifier and internal identifier of the point data in the graph data; and constructs a trie based on the external identifier and internal identifier of the point data. Based on the constructed trie, the mapping relationship between the internal identifier and external identifier of the point data can be quickly obtained, thereby solving the problems of low efficiency, long time and large disk or memory consumption when graph data is put into the database or memory, and thus improving the performance of data import.

[0093] Example 3

[0094] Figure 7A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0095] like Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0096] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0097] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the trie construction method.

[0098] In some embodiments, the trie construction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the trie construction method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to execute the trie construction method by any other suitable means (e.g., by means of firmware).

[0099] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0100] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0101] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0103] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0104] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0105] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0106] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for constructing a trie, characterized in that, include: Retrieve the external and internal identifiers of point data in the graph data; Construct a trie based on the external identifier and the internal identifier of the point data; Obtain the attribute information of point data, the external identifier of the starting point of edge data, the external identifier of the ending point of edge data, and the attribute information of edge data in graph data; A point data file is generated based on the external identifier, the internal identifier, and the attribute information of the point data, and then stored on the disk; The internal identifiers of the starting and ending points of the edge data are obtained by querying the trie based on the external identifiers of the starting and ending points of the edge data. An edge data file is generated based on the internal identifier of the starting point of the edge data, the internal identifier of the ending point of the edge data, and the attribute information of the edge data, and then stored on the disk; The dictionary is queried based on the external identifiers of the starting and ending points of the edge data to obtain the internal identifiers of the starting and ending points of the edge data, including: The trie is encoded to obtain the target data structure information; Based on the external identifiers of the starting point and the ending point of the edge data, the target data structure information is queried to obtain the internal identifiers of the starting point and the ending point of the edge data. Encoding the trie yields the target data structure information, including: The pop order of characters for external identifiers is determined based on the trie. The write position information of each character of the external identifier is determined according to the pop order of the characters of the external identifier; The node attribute information, the internal identifier of the node, and the next character are determined based on the trie; The target data structure information is determined based on the writing position information of each character, the node attribute information corresponding to each character, the internal identifier of the node, and the next character.

2. The method according to claim 1, characterized in that, Constructing a trie based on the external identifier and the internal identifier of the point data includes: The nodes of the trie are constructed based on each character of the external identifier of the point data, the position information of each character, and the internal identifier of the point data; The edges of the trie are constructed based on the external identifiers of the point data for each character and the positional relationships between the characters; Construct a trie based on the nodes and edges of the trie.

3. The method according to claim 2, characterized in that, The nodes of the trie are constructed based on each character of the external identifier of the point data, the position information of each character, and the internal identifier of the point data, including: The internal identifier of the node corresponding to the Nth character is determined based on the Nth character of the external identifier of the point data, the position information of the Nth character, and the internal identifier of the point data, where N is a positive integer greater than or equal to 1; The nodes of the trie are constructed based on the internal identifier corresponding to each node.

4. The method according to claim 3, characterized in that, The internal identifier of the node corresponding to the Nth character is determined based on the Nth character of the external identifier of the point data, the position information of the Nth character, and the internal identifier of the point data, including: If the Nth character is determined to be a non-ending character based on its position information, then the internal identifier of the node corresponding to the Nth character is determined to be the first value. If the position information of the Nth character determines that the Nth character is the ending character of the target external identifier, then the internal identifier of the node corresponding to the Nth character is determined to be the internal identifier corresponding to the target external identifier.

5. The method according to claim 1, characterized in that, A point data file is generated based on the external identifier, the internal identifier, and the attribute information of the point data, and stored on the disk, including: The key value of the point data is determined based on the external identifier and the internal identifier of the point data. The value of the point data is determined based on the external identifier and attribute information of the point data. Generate a key file based on the key value of the point data; Generate a value file based on the value values ​​of the point data; Store the key file and value file to the disk.

6. A trie construction device, characterized in that, include: The acquisition module is used to acquire the external identifier and internal identifier of point data in the graph data; The construction module is used to construct a trie based on the external identifier and the internal identifier of the point data; The trie construction device is also used for: Obtain the attribute information of point data, the external identifier of the starting point of edge data, the external identifier of the ending point of edge data, and the attribute information of edge data in graph data; A point data file is generated based on the external identifier, the internal identifier, and the attribute information of the point data, and then stored on the disk; The internal identifiers of the starting and ending points of the edge data are obtained by querying the trie based on the external identifiers of the starting and ending points of the edge data. An edge data file is generated based on the internal identifier of the starting point of the edge data, the internal identifier of the ending point of the edge data, and the attribute information of the edge data, and then stored on the disk; The dictionary is queried based on the external identifiers of the starting and ending points of the edge data to obtain the internal identifiers of the starting and ending points of the edge data, including: The trie is encoded to obtain the target data structure information; Based on the external identifiers of the starting point and the ending point of the edge data, the target data structure information is queried to obtain the internal identifiers of the starting point and the ending point of the edge data. Encoding the trie yields the target data structure information, including: The pop order of characters for external identifiers is determined based on the trie. The write position information of each character of the external identifier is determined according to the pop order of the characters of the external identifier; The node attribute information, the internal identifier of the node, and the next character are determined based on the trie; The target data structure information is determined based on the writing position information of each character, the node attribute information corresponding to each character, the internal identifier of the node, and the next character.

7. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the trie construction method according to any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the trie construction method according to any one of claims 1-5.

Images