Data partition information processing method and device, equipment and storage medium

Abstract

The invention provides a data partition information processing method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the fields of big data, intelligent search, information flow and the like. According to the specific implementation scheme, based on a preset partition hierarchy, tree nodes corresponding to partition fields of the partition hierarchy in a dictionary tree are determined; and determining child nodes of the tree nodes based on the values of the tree nodes. In the embodiment of the invention, the partition information can be managed based on the compressed dictionary tree structure, the value of the partition field is compressed to one node in the dictionary tree, the value of the partition field is processed through the path from the root node to a certain node in the dictionary tree, and the partition information can be greatly compressed; and the pressure on a database or a file system is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to the fields of big data, intelligent search, and information flow. Background Art

[0002] With the continuous development of the information age, the methods for processing data are becoming more diverse and flexible.

[0003] In the related art, business tables storing large amounts of data are usually created as partition tables or partition files. Reasonable design and use of partitions can improve query performance, simplify data management, and support more flexible data access and operations. Summary of the invention

[0004] The present disclosure provides a data partition information processing method, device, equipment and storage medium.

[0005] According to one aspect of the present disclosure, a method for processing partition information of data is provided, comprising:

[0006] Based on the preset partition level, determine the tree node corresponding to the partition field of the partition level in the dictionary tree;

[0007] Determine the child nodes of a tree node based on the value of the tree node.

[0008] According to another aspect of the present disclosure, there is provided a data partition information processing device, comprising:

[0009] A first determination module is used to determine the tree node corresponding to the partition field of the partition level in the dictionary tree based on a preset partition level;

[0010] The second determination module is used to determine the child nodes of the tree node based on the value of the tree node.

[0011] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0012] at least one processor; and

[0013] a memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0015] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0017] In the disclosed embodiment, partition information can be managed based on a compressed dictionary tree structure, the value of the partition field can be compressed into a node in the dictionary tree, and the value of the partition field can be processed using the path from the root node to a certain node in the dictionary tree. This can greatly compress the partition information and reduce the pressure on the database or file system.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 is a flowchart of a method for processing partition information of data according to an embodiment of the present disclosure;

[0021] Figure 2 is a schematic diagram of a process of determining a child node of a tree node according to an embodiment of the present disclosure;

[0022] Figure 3 It is a schematic diagram of a structure of establishing a child node corresponding to a value of a tree node according to a comparison result according to an embodiment of the present disclosure;

[0023] Figure 4 It is another structural schematic diagram of establishing a child node corresponding to the value of a tree node according to a comparison result according to an embodiment of the present disclosure;

[0024] Figure 5 It is another structural schematic diagram of establishing a child node corresponding to the value of a tree node according to a comparison result according to an embodiment of the present disclosure;

[0025] Figure 6 It is another structural schematic diagram of establishing a child node corresponding to the value of a tree node according to a comparison result according to an embodiment of the present disclosure;

[0026] Figure 7 is a schematic diagram of a shared sub-node structure according to an embodiment of the present disclosure;

[0027] Figure 8 is a structural schematic diagram of a data partition information processing device according to an embodiment of the present disclosure;

[0028] Fig. 9It is a block diagram of an electronic device used to implement the data partition information processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0030] The terms "first", "second", etc. in this disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, including a series of steps or units. The method, system, product or device is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] With the increase and accumulation of data business volume, the number of newly added partitions per day is very large, reaching hundreds of thousands or more, which makes the scale of partition information reach billions.

[0032] In the related art, for data storage, a partition data is usually stored as a piece of data in the partition table of the database; or the corresponding partition information is stored in the file directory of the corresponding table in the form of a file, and different partitions are stored in different files. When querying and using the partition information, just read the corresponding file. However, in these data processing methods, as the partition data increases, the pressure on the database and service is immeasurable, which will lead to unstable services. Similarly, as the partition data increases, the number of storage partition data tables or partition files will also increase. In addition, when all partition data needs to be read, all file information will be traversed and read or multiple partition data tables will be read, which will also cause the service response to be too slow, and also bring considerable pressure to the service system.

[0033] In view of this, an embodiment of the present disclosure provides a data partition information processing method. The method compresses the value of the partition field into one or more nodes in the dictionary tree through the data structure of the dictionary tree, and completes the storage and reading of the partition path through the path from the root node to a certain node in the dictionary tree.

[0034] like Figure 1 FIG. 1 is a flow chart of a method for processing partition information of data proposed in an embodiment of the present disclosure, including:

[0035] S101, based on a preset partition level, determining a tree node corresponding to a partition field of the partition level in the dictionary tree.

[0036] That is, the position of each partition field in the dictionary tree needs to be determined according to the partition strategy and hierarchy definition of the data. For example, when the partition hierarchy is defined according to time and geographic location, the first partition hierarchy of the partition field in the dictionary tree corresponding to the node can be time, and the second partition hierarchy can be region. The dictionary tree is constructed according to these partition hierarchies. The dictionary tree is a tree structure used to efficiently store and retrieve string sets, where each node represents a prefix of a string. In the data partitioning scenario, each node can represent a complete partition hierarchy, or multiple nodes can be used to express a complete partition hierarchy.

[0037] S102: Determine the child nodes of the tree node based on the value of the tree node.

[0038] In a dictionary tree, each tree node may contain multiple child nodes. Each child node represents a possible value, which can be any type of data such as numbers, strings, objects, etc. For example, a "time" node may have one or more child nodes such as years. A "geographic location" node may have one or more city child nodes with specific place names.

[0039] To determine the child nodes of a tree node, you need to determine it based on the value represented by the node and the actual partitioning of the data. For example, if the current node represents "time" and the data is partitioned by year, such as "2017", "2018", etc., they are all child nodes of the "year" node.

[0040] In the disclosed embodiment, based on the preset partitioning hierarchy, by organizing the data into a dictionary tree structure according to the partition fields, it is made more convenient to add, delete and modify data. When querying, it is only necessary to traverse the tree nodes related to the query conditions to obtain the dictionary tree path, which can greatly reduce the amount of data that needs to be scanned, thereby speeding up data retrieval. Partitioning and storing data in a dictionary tree structure can make data access more centralized, which is conducive to the optimization of data caching and pre-reading mechanisms, reduces the pressure on the server, and improves system performance. As the amount of data grows, the system can expand the dictionary tree horizontally or vertically by adding more nodes without affecting the access and processing of existing data. This flexible expansion method helps the system to operate stably in the long term.

[0041] In the embodiment of the present disclosure, the specific implementation steps of determining the child nodes of the tree node based on the value of the tree node are as follows: Figure 2 As shown, including the following:

[0042] S201, comparing the value of the tree node with the value of the reference node at the same partition level to obtain a comparison result.

[0043] After determining the value of each tree node, these values ​​are compared with the values ​​of other tree nodes at the same partition level (i.e., the same level). The purpose of the comparison is to find the similarities or differences between these nodes.

[0044] S202: Create a child node corresponding to the value of the tree node according to the comparison result.

[0045] After the comparison is completed, a series of comparison results will be obtained. These results may include the same child nodes, differences or a certain degree of similarity between nodes. The child nodes corresponding to the values ​​of the tree nodes can be established based on the comparison results.

[0046] In the disclosed embodiment, by comparing the values ​​of tree nodes with tree nodes at the same partition level and establishing child nodes based on the comparison results, a structured tree diagram can be constructed to clearly store and sort out the relationships between the nodes, thereby reducing storage resources, reducing resource consumption during queries, and speeding up queries.

[0047] In the disclosed embodiment, a child node corresponding to the value of the tree node can be established according to the comparison result. Specifically, the comparison result may include whether the value of the tree node is consistent with the value of the tree node at the same partition level, and whether the values ​​of the child nodes under the tree node and the tree node at the same partition level need to be split.

[0048] The following describes how to establish a dictionary tree path based on the comparison results in different situations:

[0049] (1) Case 1

[0050] When the tree node and the reference node have the same first child node value and the first child node value does not need to be split into different child nodes, a first shared child node is constructed based on the first child node value; and a dictionary tree path is created between the tree node, the reference node and the first shared child node.

[0051] Comparing the first child node values ​​of the tree node and the reference node, for example, can be implemented as follows: if the first child node values ​​of the two nodes are the same and this value does not need to be further split, that is, no finer-grained child nodes need to be created based on this value, then it can be determined that the child nodes corresponding to this value can be shared.

[0052] like Figure 3 As shown, taking the data "day=2017 / city=hainan" and "day=2018 / city=hainan" as examples, the data stored in both data are "hainan", so the child node can be used as the first shared child node.

[0053] After determining the child node corresponding to the value of the tree node, it is necessary to create paths from the tree node and the reference node to the first shared child node. Figure 3 As shown, "root" represents the root node and does not store any information. The two different partition fields "day=" and " / city=" are respectively used as separate nodes, and "city" is a child node under the "day" node. The tree node with the value of "2017" under the "day" node and the reference node with the value of "2018" are both connected to the first shared child node "hainan", so that the tree node and the reference child node have a path pointing to the first shared child node, thereby completing the construction of the dictionary tree path and realizing the compression of partition information based on the dictionary tree, thereby reducing the consumption of storage resources and being able to accurately express the partition path.

[0054] like Figure 4 As shown, taking the data "day=2017 / city=hainan", "day=2017 / city=guiyang", and "day=2018 / city=hainan" as examples, the reference node "2017" has added node data with the value "guiyang" compared to the tree node "2018". Therefore, the sub-node value of the " / city" node under the reference node "2017" is divided into two sub-nodes to store the corresponding data, and the " / city" node under the tree node "2018" continues to reuse the sub-node under the reference node "2017".

[0055] After determining the child nodes corresponding to the value of the tree node, starting from the "root" node, according to the partition information to be processed, the partition value is a tree node, and the dictionary tree path to the child nodes is established from top to bottom for the tree nodes and reference nodes. Figure 4 shown.

[0056] Among them, "day=" and " / city=" are used as separate nodes to represent different levels, avoiding the situation where each character represents a level and the dictionary tree is too deep. At the same time, "day=" and " / city=" are used as separate nodes to organize the dictionary tree structure. When the same partition value is used, it does not need to be stored repeatedly, but can be directly reused, thereby saving storage resources.

[0057] In the disclosed embodiment, when the tree node and the reference node have the same first child node value, and the first child node value does not need to be split into different child nodes, a first shared child node is constructed based on the first child node value, so as to construct a dictionary tree to process partition information, thereby reducing the pressure on the database or file system and avoiding the need to consume a large amount of storage resources to store redundant partition information. For example, for a database, multiple rows of data that originally need to be stored can be reduced to storing only one row of data, thereby saving storage space. At the same time, when querying, the child node only needs to be accessed once to obtain the required information, which reduces unnecessary data access and improves query efficiency.

[0058] (2) Case 2

[0059] When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, and the first child node value of the tree node does not need to be split into multiple child nodes, create a child node described by the first child node value for the tree node. Establish a dictionary tree path from the tree node to the child node.

[0060] Compare the first child node values ​​of the tree node and the reference node. When the first child node values ​​of the two nodes are the same, but the first child node value of the reference node needs to be further split, and the first child node value of the tree node does not need to be split, the tree node and the reference node cannot share the first child node value, so a separate child node is created for the tree node.

[0061] like Figure 5 As shown, taking the data "day=2017 / city=guiyang", "day=2017 / city=hainan", "day=2017 / city=haikou", "day=2018 / city=hainan" as an example, the values ​​of " / city" corresponding to the reference node "2017" are "hainan" and "haikou", and "hai" needs to be taken as a parent node to continue splitting the two child nodes "kou" and "nan". The value of " / city" of the tree node "2018" is "hainan", which does not need to be split. In this case, if the " / city" node under the tree node "2018" continues to reuse the "hainan" in the " / city" node under the reference node "2017", it is easy to cause inaccurate partition information. To ensure the integrity of the partition, the " / city" node under the tree node "2018" should not reuse the "hai" child node under the reference node "2017", but use "hainan" as a complete node description to create a child node of the tree node, that is, the "hainan" node under the "2018" path.

[0062] After determining the child nodes corresponding to the value of the tree node, starting from the "root" node, according to the partition information to be processed, the partition value is a tree node, and the dictionary tree path to the child nodes is established from top to bottom for the tree nodes and reference nodes. Figure 5 shown.

[0063] It should be noted that the tree nodes and reference nodes in the embodiments of the present disclosure are relative. Figure 5 As shown, if " / city" under the "2018" branch is a reference node, and " / city" under the "2017" score is a tree node, although the tree node and the reference node have the same value "hainan", "hainan" needs to be further split under the "2017" path, but does not need to be further split under the "2018" path. Therefore, the tree node is classified into multiple child nodes according to the required splitting method.

[0064] In the disclosed embodiment, when a tree node and a reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, and the first child node value of the tree node does not need to be split into multiple child nodes, a child node described by the first child node value is created separately for the tree node to construct a dictionary tree, which can further ensure the accuracy of the partition path while optimizing the use of storage and memory. By storing these data in the corresponding child nodes, storage space and memory can be used more efficiently, especially when the amount of data is large, thereby reducing system pressure.

[0065] (3) Case 3

[0066] When a tree node and a reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are different, the first child node value of the tree node is split into multiple child nodes; a dictionary tree path is established between the tree node and the multiple child nodes split from the first child node value of the tree node.

[0067] Compare the first child node values ​​of the tree node and the reference node. The first child node values ​​of the two nodes are the same, but the child nodes split from the first child node values ​​of the two nodes are different. In order to ensure the accuracy of the partition information, the same child node cannot be reused, so the child nodes with the same value are no longer reused. For example, the values ​​of the " / city" node in the "2017" branch are "hainan" and "haikou", and the values ​​of the " / city" node in the "2018" branch are "hainan" and "haiyang". Obviously, under the "2017" branch, three child nodes "hai", "kou", and "nan" need to be split, while under the "2018" branch, three child nodes "hai", "nan", and "yang" need to be split. Obviously, the three child nodes split are different. In order to ensure the accuracy of the partition information, the " / city" node of the "2017" branch no longer shares the child node "hai" with the " / city" node of the "2018" branch.

[0068] In the embodiment of the present disclosure, when a tree node and a reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split out are different, the first child node value of the tree node is split into multiple child nodes to construct a dictionary tree. By subdividing the data into multiple child nodes, it is possible to ensure that the data partition is more refined, thereby improving query performance and data management efficiency. This structure can maintain stable system performance while being more easily expanded to larger data sets.

[0069] (4) Case 4

[0070] When a tree node and a reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are the same, establish a second shared child node corresponding to the highest-level child node among the multiple split child nodes; and create a dictionary tree path between the tree node, the reference node and the second shared child node.

[0071] Compare the first child node values ​​of the tree node and the reference node. If the first child node values ​​of the two nodes are the same, the child nodes split from the first child node values ​​of the two nodes are the same. In order to ensure the accuracy of the partition information and save storage resources, the same child node can be reused. Figure 6As shown, the values ​​of the " / city" node in the "2017" branch are "guiyang", "hainan" and "haikou", and the values ​​of the city node in the "2018" branch are "hainan" and "haikou". Obviously, under the "2017" branch and the "2018" branch, the " / city" node needs to be split into three child nodes "hai", "kou" and "nan", and the "hai" node has the highest level. Therefore, the " / city" node in the "2017" branch and the " / city" node in the "2018" branch share the "hai" node.

[0072] After determining the child nodes corresponding to the value of the tree node, starting from the "root" node, according to the partition information to be processed, the partition value is a tree node, and the dictionary tree path to the child nodes is established from top to bottom for the tree nodes and reference nodes. Figure 6 shown.

[0073] In the disclosed embodiment, when a tree node and a reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple split child nodes are the same, a second shared child node corresponding to the highest-level child node among the multiple split child nodes is established to construct a dictionary tree, which can effectively improve the efficiency and performance of data management, while simplifying the data structure and reducing storage overhead.

[0074] In the disclosed embodiment, in order to ensure that the partition information is greatly compressed and to avoid the constructed dictionary tree from being too deep, a node is represented by a string in a longer partition field as much as possible. When the value of the first child node of the tree node is different from the value of the second child node of the tree node, and the value of the first child node and the value of the second child node have the same string starting from the first character, the same string is split into child nodes; the remaining string is used as the value of the child node.

[0075] by Figure 6 For example, "hainan" and "haikou" can use the same string "hai" as the parent node, and use the remaining different strings as the values ​​of the "hai" node to obtain the child nodes of the hai node, and then split downward into two independent child nodes. Similarly, the child nodes under the "day=" node are also constructed based on the same splitting method.

[0076] In the disclosed embodiment, when the values ​​of the first child node and the second child node are the same starting from the first character, the same part is split out as an independent child node, and the remaining character string is used as the value of the subsequent child node. By sharing the child nodes with the same prefix, the depth of the dictionary tree can be compressed and storage resources can be saved. In addition, the number of comparisons in the search process can be reduced, and the query efficiency can be improved. Sharing the same prefix can avoid repeated storage of the same data in the memory. This can greatly reduce the required storage space for a large amount of string data with the same prefix and improve memory usage efficiency.

[0077] In the embodiment of the present disclosure, an end identifier indicating the partition information at any level is added to the end node of the partition information at any level or the start node of the partition information at the next level.

[0078] like Figure 6 As shown in the figure, the partition fields and partition values ​​are connected in the partition order, and the slash " / " is used as the separator between the partition fields. For example, in " / city", the " / " indicates the change of the partition level, that is, the previous partition has ended and switched from one partition field to another.

[0079] In the disclosed embodiment, adding an end identifier representing any level of partition information to the end node of any level of partition information or the start node of the next level of partition information can ensure the integrity and accuracy of the data and facilitate data parsing and management of partition fields. Through a clear end identifier, the end position of each data partition can be clearly marked. It can help accurately identify the partition range of each data partition, so as to correctly process the data.

[0080] In the disclosed embodiment, an end indicator is added to each node of the dictionary tree to indicate whether the node represents a leaf node.

[0081] In the dictionary tree, a special end flag is usually added to identify whether a node is the end of a partition, that is, whether it is a leaf node. Leaf nodes are nodes that have no child nodes. In the dictionary tree, these nodes represent a complete partition path.

[0082] When implementing a dictionary tree, each node represents a character. In order to indicate whether a node is the end of a character, a Boolean variable can be added to each node, for example, named "isEnd". When the value of this variable is true, it means that the character corresponding to the node is the last character of a string, that is, the node is a leaf node. If "isEnd" is false, it means that the node is not the end of any string, or there are more child node branches.

[0083] In the disclosed embodiment, an end indicator is added to each node of the dictionary tree to indicate whether the node represents a leaf node. The end indicator is used to identify whether the path from the root node to the current node constitutes a complete partition path. If a node has an end indicator, it means that the path from the root node to the node corresponds to a complete partition path. During the search process, when a node is traversed and found to have an end indicator, it can be immediately determined that the complete string has been found without continuing the traversal, which can improve the search efficiency and accuracy.

[0084] In the embodiment of the present disclosure, in order to ensure improved efficiency of the system when data stored in a specific partition needs to be quickly accessed and processed, in the node structure of the dictionary tree, in addition to retaining the original pointers to the child nodes and possible end marks, it is also necessary to add a storage path for the partition information corresponding to the leaf nodes in the leaf nodes of the dictionary tree.

[0085] This path can be a database table name, a file path, or any information that can uniquely identify the data storage location.

[0086] In the embodiment of the present disclosure, by storing the storage path of the partition information in the leaf node, the physical or logical location of the data can be directly located when reading the data, thereby achieving fast access to the data.

[0087] To summarize, in the disclosed embodiment, partition information is stored in a compressible dictionary tree. On the basis of being able to accurately express the partition path, each node is expressed as much as possible using complete partition fields. In order to save storage resources, the string parts of the partition fields that can be shared are shared as much as possible.

[0088] like Figure 7 As shown, take the data "day = 2017 / city = guiyang", "day = 2017 / city = hainan", "day = 2017 / city = haikou", "day = 2018 / city = hainan", "day = 2018 / city = guiyang" as an example. The child node with the value "guiyang" in the tree node can be shared in the " / city" nodes of the "2017" and "2018" branches without causing errors and ambiguity. This effectively reduces the depth of the dictionary tree and saves storage resources.

[0089] In summary, the partition information processing method of data provided by the embodiment of the present disclosure can manage partition information based on a compressed dictionary tree structure, which greatly compresses the partition information, reduces the pressure on the database or file system, avoids the need to consume a large amount of storage resources, and reduces the need to store multiple rows of data to store one row of data, or multiple partition information files become one partition file. For example, in the case of a dictionary tree based on a mysql (a relational database) type database to store partitions, a column ID (Identification, an identifier) ​​is created in the database as a primary key, and the primary key and the identifier of the table to which the associated partition belongs constitute a unique index, and the dictionary tree of the partition information can be compressed and stored in the corresponding table field. When the table field can store all the information of the dictionary tree, in fact, only one row of data is used in the database to complete the storage of the partition information, which can effectively reduce the consumption of storage resources compared to the use of multiple rows of data in the corresponding technology, and even the situation where the data increases infinitely as the partitions increase. And improve query efficiency.

[0090] When the table field cannot store all the information of the dictionary tree due to length limitation, you can add a corresponding number of rows of data to store the dictionary tree. Each row of data stores different contents in the dictionary tree, and multiple rows of data together store a complete dictionary tree.

[0091] In the case of file-based storage, the entire compressed dictionary tree can be stored as a file, and even if the dictionary tree continues to grow, only one file can be used for storage. In this way, the number of files can be effectively reduced and the query efficiency can be improved.

[0092] Secondly, in terms of reading partition data, it can also avoid the problem of putting a lot of pressure on the service due to the large amount of partition data, which requires scanning many rows of partition data in the database or reading many partition information files on the file system.

[0093] The partition information processing method of data provided by the embodiment of the present disclosure, when a dictionary tree is stored in a database, only needs to read out the corresponding row data, rebuild it into the designed dictionary tree, and then traverse the dictionary tree to query the corresponding partition information, or all partition information. There is no need to scan the partition information in multiple rows of data. In the case of storing the dictionary tree based on a file, only the dictionary tree is rebuilt based on the file, and then traversed, without reading multiple files of partition information. Therefore, the method provided by the embodiment of the present disclosure reduces the pressure on the database and file system, and also enhances the stability and performance of the service.

[0094] Based on the same technical concept, the embodiment of the present disclosure also provides a data partition information processing device 800, such as Figure 8 As shown, including:

[0095] The first determination module 801 is used to determine the tree node corresponding to the partition field of the partition level in the dictionary tree based on the preset partition level;

[0096] The second determination module 802 is used to determine the child nodes of the tree node based on the value of the tree node.

[0097] In some embodiments, the second determining module includes:

[0098] A comparison unit, used to compare the value of the tree node with the value of the tree node at the same partition level to obtain a comparison result;

[0099] The establishing unit is used to establish a child node corresponding to the value of the tree node according to the comparison result.

[0100] In some embodiments, the establishing unit is further configured to:

[0101] In the case where the tree node and the reference node have the same first child node value and the first child node value does not need to be split into different child nodes, constructing a first shared child node based on the first child node value; and,

[0102] Create a trie path between the tree node, the reference node, and the first shared child node.

[0103] In some embodiments, the establishing unit is specifically configured to:

[0104] When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, and the first child node value of the tree node does not need to be split into multiple child nodes, create a child node described by the first child node value for the tree node;

[0105] Establish a dictionary tree path from a tree node to its child nodes.

[0106] In some embodiments, the establishing unit is specifically configured to:

[0107] When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are different, the first child node value of the tree node is split into multiple child nodes;

[0108] A dictionary tree path is established between a tree node and a plurality of child nodes split from the first child node of the tree node.

[0109] In some embodiments, the establishing unit is specifically configured to:

[0110] When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are the same, establish a second shared child node corresponding to the highest-level child node among the multiple child nodes split;

[0111] Create a trie path between the tree node, the reference node, and the second shared child node.

[0112] In some embodiments, where the first child node value of a tree node is different from the second child node value of the tree node, and the first child node value and the second child node value have the same string starting from the first character, the same string is split into child nodes; the remaining string is used as the value of the child node.

[0113] In some embodiments, the system further includes a first adding module for:

[0114] An end identifier indicating the partition information at any level is added to the end node of the partition information at any level or the start node of the partition information at the next level.

[0115] In some embodiments, a second adding module is further included, for:

[0116] At each node of the dictionary tree, an end indicator is added to indicate whether the node represents a leaf node.

[0117] In some embodiments, a storage module is further included for:

[0118] In the leaf node of the dictionary tree, add the storage path of the partition information corresponding to the leaf node.

[0119] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0120] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0121] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0122] Fig. 9A schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0123] like Fig. 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0124] A number of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0125] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as a partition information processing method for data. For example, in some embodiments, the partition information processing method for data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the partition information processing method for data described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the partition information processing method for data in any other appropriate manner (e.g., by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0131] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0132] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0133] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for processing partition information of data, comprising: Based on a preset partition level, determining a tree node corresponding to a partition field of the partition level in the dictionary tree; Based on the value of the tree node, a child node of the tree node is determined.

2. The method according to claim 1, wherein: The determining of the child nodes of the tree node based on the value of the tree node includes: Comparing the value of the tree node with the value of a reference node at the same partition level to obtain a comparison result; According to the comparison result, a child node corresponding to the value of the tree node is established.

3. The method according to claim 2, wherein: The step of establishing a child node corresponding to the value of the tree node according to the comparison result includes: In the case where the tree node and the reference node have the same first child node value and the first child node value does not need to be split into different child nodes, constructing a first shared child node based on the first child node value; and, A dictionary tree path is created between the tree node, the reference node and the first shared child node.

4. The method according to claim 2, wherein: The step of establishing a child node corresponding to the value of the tree node according to the comparison result includes: In a case where the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, and the first child node value of the tree node does not need to be split into multiple child nodes, creating the child node described by the first child node value for the tree node; A dictionary tree path is established from the tree node to the child node.

5. The method according to claim 2, wherein: The step of establishing a child node corresponding to the value of the tree node according to the comparison result includes: In the case where the tree node and the reference node have the same first subnode value, the first subnode value of the reference node needs to be split into multiple subnodes, the first subnode value of the tree node needs to be split into multiple subnodes, and the multiple subnodes split from the tree node and the reference node for the first subnode value are different, the first subnode value of the tree node is split into multiple subnodes; A dictionary tree path is established between the tree node and the multiple child nodes split from the first child node value of the tree node.

6. The method according to claim 2, wherein: The step of establishing a child node corresponding to the value of the tree node according to the comparison result includes: When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are the same, a second shared child node corresponding to the highest-level child node among the multiple child nodes split; A dictionary tree path is created between the tree node, the reference node and the second shared child node.

7. The method according to any one of claims 3 to 6, wherein: When the first child node value of the tree node is different from the second child node value of the tree node, and the first child node value and the second child node value have the same string starting from the first character, the same string is split into child nodes; the remaining string is used as the value of the child node.

8. The method according to any one of claims 1 to 7, further comprising: An end identifier indicating the partition information at any level is added to the end node of the partition information at any level or the start node of the partition information at the next level.

9. The method according to any one of claims 1 to 8, further comprising: At each node of the dictionary tree, an end indicator is added to indicate whether the node represents a leaf node.

10. The method according to any one of claims 1 to 9, further comprising: In the leaf node of the dictionary tree, add the storage path of the partition information corresponding to the leaf node.

11. A data partition information processing device, comprising: A first determination module is used to determine, based on a preset partition level, a tree node corresponding to a partition field of the partition level in the dictionary tree; The second determining module is used to determine the child nodes of the tree node based on the value of the tree node.

12. The device according to claim 11, wherein The second determining module includes: A comparison unit, used to compare the value of the tree node with the value of the reference node at the same partition level to obtain a comparison result; The establishing unit is used to establish a child node corresponding to the value of the tree node according to the comparison result.

13. The device according to claim 12, wherein: The establishing unit is specifically used for: In a case where the tree node and the reference node have the same first child node value and the first child node value does not need to be split into different child nodes, constructing a first shared child node based on the first child node value; as well as, A dictionary tree path is created between the tree node, the reference node and the first shared child node.

14. The device according to claim 12, wherein: The establishing unit is specifically used for: In a case where the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, and the first child node value of the tree node does not need to be split into multiple child nodes, creating the child node described by the first child node value for the tree node; A dictionary tree path is established from the tree node to the child node.

15. The device according to claim 12, wherein: The establishing unit is specifically used for: In the case where the tree node and the reference node have the same first subnode value, the first subnode value of the reference node needs to be split into multiple subnodes, the first subnode value of the tree node needs to be split into multiple subnodes, and the multiple subnodes split from the tree node and the reference node for the first subnode value are different, the first subnode value of the tree node is split into multiple subnodes; A dictionary tree path is established between the tree node and the multiple child nodes split from the first child node value of the tree node.

16. The device according to claim 12, wherein: The establishing unit is specifically used for: When the tree node and the reference node have the same first child node value, the first child node value of the reference node needs to be split into multiple child nodes, the first child node value of the tree node needs to be split into multiple child nodes, and the multiple child nodes split from the tree node and the reference node for the first child node value are the same, a second shared child node corresponding to the highest-level child node among the multiple child nodes split; A dictionary tree path is created between the tree node, the reference node and the second shared child node.

17. The device according to any one of claims 13 to 16, wherein: When the first child node value of the tree node is different from the second child node value of the tree node, and the first child node value and the second child node value have the same string starting from the first character, the same string is split into child nodes; the remaining string is used as the value of the child node.

18. The apparatus according to any one of claims 11 to 17, further comprising a first adding module, configured to: An end identifier indicating the partition information at any level is added to the end node of the partition information at any level or the start node of the partition information at the next level.

19. The apparatus according to any one of claims 11 to 18, further comprising a second adding module, configured to: At each node of the dictionary tree, an end indicator is added to indicate whether the node represents a leaf node.

20. The device according to any one of claims 11 to 19, further comprising a storage module, configured to: In the leaf node of the dictionary tree, add the storage path of the partition information corresponding to the leaf node.

21. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images