A fast indexing method and device based on improved B-tree search algorithm

Abstract

The present invention provides a fast indexing method and device based on an improved B-tree search algorithm, the method comprising the following steps: assigning a color attribute according to a keyword type attribute of each node in any tree data structure; establishing an index linking the current node with the next node having the same color keyword, the index including an index value, the index value being a keyword or empty. Through this method, most of the search nodes in the system are confined to the same belonging group, so if different colors are given to each object to express such attribute values, the search range can be narrowed to nodes having the same color keyword, thereby reducing the search order of the B-search tree, accelerating the search efficiency of the keywords inside the B-search tree nodes, and reducing unnecessary searches.

Description

Technical Field

[0001] The invention belongs to the technical field of fast search, and in particular relates to a fast indexing method and device based on an improved B-tree search algorithm. Background Art

[0002] As the equipment in the communication room becomes increasingly complex, the scale of time series data generated in the room has exploded, including various types of time series data in communication equipment indicators, network data, and sensor data, which are continuously collected at fixed or irregular time intervals. The structure of time series data is not complex, and its typical characteristics are high frequency of generation and large amount of data. Massive time series data is a test for storage and query capabilities.

[0003] When there are many nodes in the computer room system, the commonly used search algorithm will cause more search time to be wasted on communication between nodes, which not only leads to a serious decrease in search efficiency but also affects the network stability of the computer room system.

[0004] At present, the B-tree method is used to collect the time series data generated by the computer room. The B-tree reduces the height of the search tree so that the search can be completed with only a small amount of communication, which not only speeds up the speed but also reduces the network load. However, the essence of this method is to group more simulation nodes, which can only be roughly located in the corresponding group, and then accurately located by searching in the group. At the same time, when there are many nodes in the system, the commonly used search algorithm will waste more search time on the communication between nodes, which not only leads to a serious decrease in search efficiency but also affects the network stability of the system. Summary of the invention

[0005] The purpose of the present invention is to overcome the above problems and provide a fast indexing method and device based on an improved B-tree search algorithm.

[0006] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0007] On the one hand, a fast indexing method based on an improved B-tree search algorithm is provided, comprising the following steps:

[0008] Assign color attributes according to the keyword type attribute of each node in any tree data structure;

[0009] Establishing an index linking the current node to the next node with the same color keyword, the index includes an index value, and the index value is a keyword or empty;

[0010] When the next node with the same color keyword adjacent to the current node includes a node with a different color keyword, the keyword of the current node is the same as the index value of the next node with the same color keyword;

[0011] When the next node with the same color keyword adjacent to the current node does not include the node with a different color keyword, the index value of the next node with the same color keyword is empty.

[0012] As a further improvement, the current node is a non-leaf node at the bottom of the tree data structure, and the fast indexing method further includes the following steps of creating a new non-leaf node:

[0013] When a new keyword needs to be added, first create a new non-leaf node at the bottom layer of the non-leaf node of the tree data structure and adjacent to the right side of the current node, then insert the new keyword into the new non-leaf node, and the current node and the new non-leaf node form a member node;

[0014] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the new non-leaf node is inserted, where m is the order;

[0015] An index linking the new non-leaf node to the previous non-leaf node with the same color keyword is established.

[0016] As a further improvement, the member node also includes a left adjacent node adjacent to the left side of the current node, the keyword of the member node includes a left keyword, a middle keyword and a right keyword, and the step of creating a new non-leaf node also includes:

[0017] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0018] As a further improvement, the step of creating a new non-leaf node also includes:

[0019] If the number of keywords of the previous level member node is greater than m-1, repeat the following steps:

[0020] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the insertion of the new non-leaf node is completed;

[0021] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0022] As a further improvement, the nodes with the same color keywords in the tree data structure form a plurality of ordered sequences of the same color keywords, and the fast indexing method further includes the following keyword query step:

[0023] Selecting an ordered sequence of keywords of the same color that have the same color attribute as the keyword to be queried, and querying each node having the keyword of the same color;

[0024] If the index value of the current node with the same color keyword is empty, query the next adjacent node with the same color keyword;

[0025] If the index value of the current node with the same color keyword is the keyword, jump to the next node with the same color keyword corresponding to the keyword;

[0026] Until a node with the same color keyword as the keyword to be queried is found.

[0027] As a further improvement, the keywords of all the nodes of the tree data structure form a mixed color ordered sequence (an ordered sequence of keywords with multiple color attributes); the fast indexing method further includes the following keyword query step:

[0028] Selecting the ordered sequence of same-color keywords with different color attributes from the keyword to be queried, and finding a minimum interval in the ordered sequence of same-color keywords with different color attributes; the minimum interval is two consecutive nodes on the ordered sequence of same-color keywords with different color attributes, and a section of mixed-color ordered sequence intercepted from the mixed-color ordered sequence based on the two consecutive nodes, the section of mixed-color ordered sequence including the keyword to be queried;

[0029] The intercepted section of the mixed color ordered sequence is inserted between the two continuous nodes, and then the keyword to be queried is searched in the intercepted section of the mixed color ordered sequence.

[0030] As a further improvement, the nodes with the same-color keywords in the tree data structure form two ordered sequences of the same-color keywords.

[0031] On the other hand, a fast search method for a database is proposed, including the fast indexing method based on the improved B-tree search algorithm and a non-time series data search method. The method server includes the following steps:

[0032] Receive keyword information from the client;

[0033] Processing the keyword information into a single marked word segmentation element through lexical analysis;

[0034] Using an inverted index algorithm, matching the stored documents including the keyword in the constructed inverted index;

[0035] The stored document is sent to the client.

[0036] On the other hand, the present invention also provides a fast indexing device based on an improved B-tree search algorithm, the device comprising at least one processor and a memory storing instructions, and when the instructions are executed by at least one processor, the method in the above technical solution is implemented.

[0037] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in the steps of the above technical solution.

[0038] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method described in the steps of the above technical solution is implemented.

[0039] The beneficial effects of the present invention are:

[0040] Through this method, the keywords are accurately located, which greatly improves the search and search speed; saves the search time on the node, improves the search efficiency and the network stability of the system; adopts a search algorithm with better search efficiency and less network communication resource occupation to ensure the simulation real-time and stability of the simulation system; most of the search nodes in the system are confined to the same belonging group, so if a different color is given to each object to express such attribute values, the search range can be narrowed to the nodes with the same color keywords, thereby reducing the search order of the B search tree, accelerating the search efficiency of the keywords inside the B search tree nodes, and reducing unnecessary searches. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention has the following accompanying drawings:

[0042] Figure 1 It is a flowchart of a fast indexing method based on an improved B-tree search algorithm in some specific embodiments of the present invention;

[0043] Figure 2 is a flow chart of a method for quickly searching a database in some specific embodiments of the present invention;

[0044] Figure 3 It is a database business flow chart of some specific embodiments of the present invention;

[0045] Figure 4 is a flow chart of a Chinese word segmentation algorithm based on a hidden Markov model in some specific embodiments of the present invention;

[0046] Figure 5 is a diagram of the construction process of an improved red-blue B-tree according to some specific embodiments of the present invention;

[0047] Figure 6 is a keyword in a tree node in some specific embodiments of the present invention;

[0048] Figure 7 A schematic diagram of an improved B-tree construction device of the present invention. DETAILED DESCRIPTION

[0049] The following examples further illustrate the content of the present invention, but should not be construed as limiting the present invention. Without departing from the spirit and substance of the present invention, modifications or substitutions made to the methods, steps or conditions of the present invention all fall within the scope of the present invention.

[0050] like Figure 1 As shown, the present invention provides a fast indexing method based on an improved B-tree search algorithm, comprising the following steps:

[0051] Step S10: assigning a color attribute according to the keyword type attribute of each node in any tree data structure;

[0052] For example, in a tree data structure, each node is assigned a different color attribute. For example, the values ​​in the two classes are stored in different nodes, one node is assigned a red attribute, and the other node is assigned a blue attribute.

[0053] Step S11: establishing an index linking the current node to the next node with the same color keyword, wherein the index includes an index value, and the index value is a keyword or empty;

[0054] For example, in the nodes of the tree data structure, if there is a node interval of different color keywords between two nodes of the same color keyword, then the index value of the second node exists in the node of the first same color keyword, and the index value is equal to the keyword of the second node of the same color keyword. In this way, the two nodes of the same color keyword can be linked. Because there is an index, when searching, you can skip the nodes of different color keywords and find the elements to be found in the nodes of the same color keyword, which saves time, reduces the system's jam rate, and makes the system more stable.

[0055] After the above steps S10 and S11 are completed, one of the following steps is performed:

[0056] When the next node with the same color keyword adjacent to the current node includes a node with a different color keyword, the keyword of the current node is the same as the index value of the next node with the same color keyword;

[0057] When the next node with the same color keyword adjacent to the current node does not include the node with a different color keyword, the index value of the next node with the same color keyword is empty.

[0058] The current node is a non-leaf node at the bottom of the tree data structure, and the fast indexing method further includes the following steps of creating a new non-leaf node:

[0059] When a new keyword needs to be added, first create a new non-leaf node at the bottom layer of the non-leaf node of the tree data structure and adjacent to the right side of the current node, then insert the new keyword into the new non-leaf node, and the current node and the new non-leaf node form a member node;

[0060] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the new non-leaf node is inserted, where m is the order;

[0061] An index linking the new non-leaf node to the previous non-leaf node with the same color keyword is established.

[0062] For example, the current node is a non-leaf node at the bottom of the tree data structure, and the fast indexing method further includes the following steps of creating a new non-leaf node:

[0063] When a new keyword needs to be added, first create a new non-leaf node at the bottom layer of the non-leaf node of the tree data structure and adjacent to the right side of the current node, then insert the new keyword into the new non-leaf node, and the current node and the new non-leaf node form a member node;

[0064] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the new non-leaf node is inserted, where m is the order;

[0065] An index linking the new non-leaf node to the previous non-leaf node with the same color keyword is established.

[0066] The member node also includes a left adjacent node adjacent to the left side of the current node, the keyword of the member node includes a left keyword, a middle keyword and a right keyword, and the step of creating a new non-leaf node also includes:

[0067] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0068] For example, the member node also includes a left adjacent node adjacent to the left side of the current node, the keyword of the member node includes a left keyword, a middle keyword and a right keyword, and the step of creating a new non-leaf node also includes:

[0069] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0070] The step of creating a new non-leaf node also includes:

[0071] If the number of keywords of the previous level member node is greater than m-1, repeat the following steps:

[0072] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the insertion of the new non-leaf node is completed;

[0073] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0074] For example, the member node also includes a left adjacent node adjacent to the left side of the current node, the keyword of the member node includes a left keyword, a middle keyword and a right keyword, and the step of creating a new non-leaf node also includes:

[0075] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0076] The steps of creating a new non-leaf node also include:

[0077] If the number of keywords of the previous level member node is greater than m-1, repeat the following steps:

[0078] If the sum of the number of keywords in the member nodes is less than or equal to m-1, the insertion of the new non-leaf node is completed;

[0079] If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

[0080] For example, according to the definition rules of the red-blue B-tree above, when inserting a member node in the tree, you cannot directly add a leaf node, otherwise it will violate its definition rules. Here is the process of inserting a member node in the red-blue B-tree:

[0081] Step 1: Add a keyword to the corresponding non-leaf node at the lowest level. If the number of member keywords does not exceed m-1, where m is the order of the node, the insertion is complete, otherwise proceed to step 2.

[0082] Step 2: Split the member nodes according to the splitting rules until they meet the splitting rules defined below.

[0083] The splitting rule of the red-blue B-tree member node is: divide the keyword into two parts from the middle of the member node, the left one is smaller than the middle one, and the right one is larger than the middle one. The left keyword is placed in the original node, the right keyword is placed in the new node, and the middle keyword and the new node are inserted into the parent node. If the parent node also has more than m-1 keywords, it needs to be split again until all nodes meet the rules.

[0084] For example, Figure 5 As shown, according to the above rules, the process of constructing a red-blue B-tree of order 3 is given here. The seven nodes with keywords 1-7, the color attributes of keywords 1, 3, and 5 are red, and the color attributes of keywords 2, 4, and 6 are blue. The green key value in the figure represents the key value of the next keyword of the same color in the same node. Red: node key value of red attribute. Blue: node key value of blue attribute. Figure 5 In the figure (the process of tree structure change when inserting a new non-leaf node; the following describes the insertion of a new node as a new non-leaf node), the member node description in the first group of tree structures, the first node in the figure, has only one node, and the keyword is 1. Figure 5 In the description of the member nodes in the second group of tree structures, a blue node with a keyword of 2 is inserted into the second node (i.e. Figure 5 The second member node in the , consists of two nodes with keywords 1 and 2), the member node that makes up the two nodes (consists of nodes with keywords 1 and 2), at this time, it is judged that the condition greater than m-1 is not met; Figure 5In the description of the member nodes in the third group of tree structures, after the third node with keyword 3 is inserted, the member nodes become 3 (consisting of nodes with keywords 1, 2 and 3). If the condition greater than m-1 is found, the node needs to be split. The keyword is divided into two parts from the middle of the member node. The left part is smaller than the middle part, and the right part is larger than the middle part. The keyword on the left side of the original member node is placed in the original node (that is, Figure 5 In the third tree structure, the key 1 in the original node 1 is still placed in the left node 1 at the bottom layer), the key 3 on the right is placed in the new node, and the node in the middle including key 2 is inserted into the node of the previous level (at this time, it constitutes the first node of the member node of the previous level). Figure 5 In the description of the member nodes in the fourth group of tree structures, when a node with keyword 4 is inserted into the member including keyword 3, the member nodes composed of keywords 3 and 4 do not meet the condition greater than m-1 at this time. Therefore, after inserting the node with keyword 4, the member node does not split. Figure 5 In the description of the member nodes in the fifth group of tree structures, after inserting a node with keyword 5 into the member nodes composed of keywords 3 and 4, the member nodes composed of keywords 3 and 4 meet the condition of being greater than m-1, then this group of member nodes continues to split, the node with keyword 3 is placed in the original node, the middle node including keyword 4 and the middle node are inserted into the upper node (at this time, the nodes with keywords 2 and 4 form the member nodes), and the keyword with keyword 5 is placed in the new node. Figure 5 In the description of the member nodes in the sixth group of tree structures, when a new node with keyword 6 is inserted after the node with keyword 5, the condition greater than m-1 is not met at this time. Therefore, after the node with keyword 6 is inserted, the member node does not split. Figure 5 In the description of the member nodes in the 7th group of tree structures, when a new node with keyword 7 is inserted into the member node composed of nodes with keywords 5 and 6, the condition of being greater than m-1 is met at this time, and the member nodes composed of nodes with keywords 5, 6 and 7 are to be split, and the keyword with keyword 5 is placed in the original node of the member node composed of nodes with keywords 5, 6 and 7, the keyword 6 in the middle position and the middle node are inserted into the member node of the upper level, and the keyword with keyword 7 is placed in the new node; at this time, when the keyword 6 in the middle position and the middle node are inserted into the member node of the upper level, the member node of the upper level becomes a member node composed of nodes with keywords 2, 4 and 6, and the condition of being greater than m-1 is also met at this time, because the member nodes also need to be split, and the node with keyword 2 is still placed in the original node with keyword 2, the keyword 4 in the middle position and the node in the middle position are inserted into the node of the next higher level, and the keyword with keyword 6 is still placed in the original node with keyword 6.

[0085] Step 3: Create a new index for keywords of the same color in related nodes, that is, the previous keyword of the same color should contain the information of the next keyword of the same color.

[0086] In some embodiments of the present invention, the keywords of all the nodes of the tree data structure form a mixed color ordered sequence; the fast indexing method further includes the following keyword query step:

[0087] Selecting the ordered sequence of same-color keywords with different color attributes from the keyword to be queried, and finding a minimum interval in the ordered sequence of same-color keywords with different color attributes; the minimum interval is two consecutive nodes on the ordered sequence of same-color keywords with different color attributes, and a section of mixed-color ordered sequence intercepted from the mixed-color ordered sequence based on the two consecutive nodes, the section of mixed-color ordered sequence including the keyword to be queried;

[0088] The intercepted section of the mixed color ordered sequence is inserted between the two continuous nodes, and then the keyword to be queried is searched in the intercepted section of the mixed color ordered sequence.

[0089] In some embodiments of the present invention, the keywords of all nodes of the tree data structure form a mixed color ordered sequence (an ordered sequence of keywords with multiple color attributes, an ordered sequence of keywords composed of keywords with multiple color attributes);

[0090] The fast indexing method also includes the following keyword query step (this method is applied to the situation where there are keyword nodes with multiple different colors from the keyword to be queried between the minimum intervals, when the keyword is first searched in a keyword sequence, the keyword to be searched is not found, because the color of the keyword to be queried is not clear at the beginning, only a keyword sequence of one color is used to search, and only the keyword sequence interval with the keyword is found; the minimum interval is obtained, and then other nodes between the minimum intervals are inserted into it (and then a mixed color ordered sequence is intercepted from the mixed color ordered sequence, and then inserted between two consecutive nodes in the minimum interval), the minimum value of the ordered sequence composed of the keyword ordered sequences of these other nodes is greater than the minimum value of the minimum interval, and the maximum value of the ordered sequence is smaller than the maximum value of the minimum interval. Then the keyword to be queried is searched in the ordered sequence, and the keyword to be queried is searched by the indexing method in the following steps H31 and H32):

[0091] H1: selecting the ordered sequence of the same color keywords having the color attribute different from the keyword to be queried, and finding the minimum interval in the ordered sequence of the same color keywords having the different color attributes;

[0092] H2: inserting into the minimum interval an ordered sequence of the mixed color ordered sequence including the keyword to be queried located between the two adjacent continuous nodes;

[0093] H3: Then, in the intercepted mixed color ordered sequence, the keyword to be queried is searched; wherein,

[0094] In the intercepted mixed color ordered sequence, the keyword to be queried is searched and the following steps are performed:

[0095] H31: If the index value of the current node with the same color keyword is empty, query the next adjacent node with the same color keyword;

[0096] H32: If the index value of the current node with the same color keyword is the keyword, jump to the next node with the same color keyword corresponding to the keyword;

[0097] Until a node with the same color keyword as the keyword to be queried is found.

[0098] For example, because the color of the keyword is not known in advance, when the query keyword sequence is different from the keyword to be queried, the method of "if the index value of the current node with the same color keyword is empty, query the adjacent next node with the same color keyword; if the index value of the current node with the same color keyword is the keyword, jump to the next node with the same color keyword corresponding to the keyword" is adopted, and then find the interval containing the keyword sequence to be queried (the color of the ending keyword of the interval is different from the color of the keyword, which becomes the minimum interval), insert multiple nodes into the minimum interval, and generate an ordered sequence of the minimum intervals with the same color attribute as the keyword to be queried. Finally, in the ordered sequence, query the keyword through the above-mentioned method of "if the index value of the current node with the same color keyword is empty, query the adjacent next node with the same color keyword; if the index value of the current node with the same color keyword is the keyword, jump to the next node with the same color keyword corresponding to the keyword" is adopted. In this way, within the framework of the minimum interval, by inserting the node of the keyword, the keywords outside the minimum interval that are the same as the keyword to be queried do not need to be queried again, which can improve the query efficiency.

[0099] In some embodiments of the present invention, the nodes with the same color keywords of the tree data structure form two ordered sequences of the same color keywords (for example, the color attributes of the ordered sequences of the same color keywords include red and blue; wherein the color attribute of the ordered sequences of the same color keywords with different color attributes is red, and the color attribute of the ordered sequences of the same color keywords with the same color attribute is blue).

[0100] For example, the keyword query method further includes the following steps:

[0101] Inserting a mixed color ordered sequence into the minimum interval to generate the intercepted mixed color ordered sequence;

[0102] In the intercepted mixed color ordered sequence, the keyword to be queried is searched; wherein,

[0103] In the intercepted mixed color ordered sequence, the keyword to be queried is searched, including the following steps (this method is applied to the situation where there are keyword nodes with the same color as the keyword to be queried between the minimum intervals. When searching for the keyword in a keyword sequence first, the keyword to be queried is not found because the color of the keyword to be queried is not clear at the beginning. Only a keyword sequence of one color is used for searching, and only the keyword sequence interval with the keyword is found; the minimum interval is obtained, and then other nodes between the minimum interval are inserted into it. The minimum value of the ordered sequence composed of the keyword ordered sequences of these other nodes is greater than the minimum value of the minimum interval, and the maximum value of the ordered sequence is smaller than the maximum value of the minimum interval. Then the keyword to be queried is searched in the ordered sequence, and the keyword to be queried is searched by repeating the following steps. There are only two colors for this search. The keyword ordered sequence interval searched for the first time is a keyword ordered sequence interval of a different color from the keyword to be queried, and the interval searched for the second time is exactly all keyword ordered sequences of the same color as the keyword to be queried):

[0104] When the index value of the current node with the same color keyword is found to be empty, the next adjacent node with the same color keyword is queried;

[0105] Until a node with a keyword equal to the keyword to be queried is found.

[0106] For example, an interval composed of the color attributes different from the keyword to be queried is the minimum interval (the color of the keyword to be queried is not known for the first time. After a node sequence of the same color keyword with different color attributes from the keyword to be queried is used, the keyword to be queried is searched in this node sequence of the same color keyword with different color attributes from the keyword to be queried, and finally a minimum sequence interval including the keyword to be queried is found in this node sequence of the same color keyword with different color attributes from the keyword to be queried); an interval composed of nodes of an ordered sequence added to the minimum interval is an ordered sequence (after finding the minimum interval, multiple keyword nodes are added to the minimum interval. Since all keywords are arranged in order, the keywords of the nodes in the entire tree structure are also arranged in order. For example, the keywords are 1, 2, 3... or 1, 3, 6, 7, 8..., all arranged in order from small to large).

[0107] In some embodiments of the present invention, in the nodes having the same color keyword, the previous same color keyword includes information of the next same color keyword.

[0108] For example, a new index is created for keywords of the same color in related nodes, that is, the previous keyword of the same color should contain the information of the next keyword of the same color.

[0109] In some embodiments of the present invention, the following steps are also included:

[0110] Start searching from the root node and search for a keyword sequence in a node with the same color keyword; if it can be found, return the keyword and end;

[0111] Otherwise, search in the node of the interval to which the keyword belongs;

[0112] Repeat the recursion until the keyword is found or a leaf node is found.

[0113] In some embodiments of the present invention, the keywords in the tree nodes of the red-blue tree data structure are arranged in order, and the keywords of the same color are also arranged in order. Therefore, a sequential search or a binary search can be used to search for keywords within the same color range.

[0114] like Figure 6As shown, the arrangement and search process of keywords in the nodes shown in the following table are given here: if the color after the keyword is the same, the index is 0, otherwise it is the keyword information of the next keyword of the same color. For example, the nodes in the first row of the table below, the first, fourth and second rows are all red attribute values, and the remaining nodes are blue attribute values. The 2 keyword node is followed by a blue node, that is, the index value of the next red node 7 is set after the 2 keyword. Assuming that you want to find the red 17 keyword, you only need to search the 6 keywords 2, 7, 17, 19, 21, 22 in sequence through a binary search. If you want to find the minimum belonging interval, after finding the minimum interval of the red keyword, introduce the blue node to further refine the interval. The search result is a successful comparison or finding the minimum belonging interval. At this time, in order to narrow the belonging interval of the keyword, you can use nodes with different color keywords to insert.

[0115] In some embodiments of the present invention, the tree data structure further includes:

[0116] The root node can be a single leaf node;

[0117] If the root node is a non-leaf node, then there are at least 2 child nodes;

[0118] The number of child nodes is less than or equal to the order;

[0119] The non-leaf nodes include at least m / 2 whole children upwards, where m is the order;

[0120] The keywords in the non-leaf nodes are ordered sequences;

[0121] When all leaf nodes are at the same level, they can all be replaced by NULL.

[0122] For example, the rules of the m-order red-blue B-tree are given here with reference to the rules of the B-tree, where n is the total number of nodes and m is the order of the red-blue B-tree.

[0123] 1. The root node can be a single leaf node;

[0124] 2. The root node should have at least 2 or more child nodes;

[0125] 3. The number of child nodes of all member nodes in the red-blue B-tree should not exceed the order;

[0126] 4. Non-leaf nodes should have at least m / 2 whole children;

[0127] 5. The keywords in non-leaf nodes are ordered sequences;

[0128] 6. All leaf nodes are located in the same layer and can be replaced by NULL;

[0129] 7. All nodes have red or blue attributes;

[0130] 8. All keywords should be linked to the next keyword of the same color.

[0131] On the other hand, a fast search method for a database is proposed, including the fast indexing method based on the improved B-tree search algorithm and a non-time series data search method. The method server includes the following steps:

[0132] Step S100: receiving keyword information from a client;

[0133] Step S101: Processing the keyword information into a single marked word segmentation element through lexical analysis;

[0134] Step S102: using an inverted index algorithm to match a stored document including the keyword in the constructed inverted index;

[0135] Step S103: Send the stored document to the client.

[0136] For example, when a user enters keyword information, the keyword is first processed into word segmentation elements of tokens through lexical analysis, and then the inverted index algorithm is used to quickly search for documents containing the word segmentation elements, quickly match them in the constructed inverted index, and finally display the search results to the user.

[0137] like Figure 3 As shown, the server also includes the following steps:

[0138] Step S20: collecting data;

[0139] Step S21: sending the collected data to the calibration index to create a document set;

[0140] For example, the first step is to create a document. After obtaining the collected data, the database sends it to the specified index to create a new document set.

[0141] Step S22: using a hidden Markov model word segmentation method to perform word segmentation processing on the document set, and breaking it into single word segmentation elements;

[0142] For example, the second step is to analyze the newly created document set (word segmentation), that is, the text must be broken down into a series of word segmentation elements called tokens.

[0143] Step S23: creating an inverted index for the word segmentation element;

[0144] For example, after word segmentation, the database will create an inverted index for each word, recording in which documents the word appears and where it appears.

[0145] Step S24: adding the document including the word segmentation element to the inverted index.

[0146] For example, documents are added to the index, and data is written to the storage.

[0147] In some embodiments of the present invention, the inverted index includes:

[0148] Word dictionary, inverted list.

[0149] In some embodiments of the present invention, the word dictionary includes:

[0150] Document number, document content; wherein the document content includes words of the document.

[0151] Step S22 includes the following steps:

[0152] Step S220: using statistical word segmentation to estimate the state transition probability and the probability of observing a specific Chinese character in each state by training a large amount of annotated corpus data;

[0153] Step S221: Based on the initial state probability, the state transition probability and the observation probability, a dynamic programming algorithm is used to retain the local optimal path during the calculation process to obtain a globally optimal word segmentation result.

[0154] Step S23 includes the following steps:

[0155] Step S230: using word segmentation technology to split the content of each document into separate words;

[0156] Step S231: creating a sorted list containing all the non-repeating individual words;

[0157] Step S232: List the documents including the individual words and the position information in the documents.

[0158] In the present invention, the database used has distributed real-time document storage capabilities, each field can be indexed and searched, and supports database expansion and mass data management.

[0159] 1. Database cluster technology

[0160] The database adopts a cluster architecture, which consists of multiple nodes. By distributing and storing data on multiple nodes, it not only improves the availability of the system, but also ensures the reliability and fault tolerance of the data, so that the database can be easily expanded to multiple nodes to cope with the growing amount of data. The cluster is responsible for the storage and search functions of the data, and the nodes communicate through a distributed protocol. The cluster can be expanded horizontally. By adding additional nodes to the cluster, the capacity of the cluster can be increased linearly while maintaining similar performance. The node is the basic unit of the cluster, and each node contains one or more indexes. The node can be configured with different roles, such as master node, data node, coordination node, client node, ingestion node, etc.

[0161] (1) Master Node: Each cluster has a single master node at any time, whose responsibilities include maintaining the health of the cluster. They are usually lightweight because they do not process any data. The master node acts as a coordinator to create, delete, and manage indexes on appropriate nodes in the cluster, and to assign indexes and underlying shards to appropriate nodes.

[0162] (2) Master-Eligible Nodes: Master-Eligible Nodes are nodes that may become master nodes. It is recommended to build a cluster with more than one master-eligible node, and the number of master-eligible nodes should be an odd number to handle cluster partitions.

[0163] (3) Data Nodes: Data nodes store the actual index data and handle data ingestion, search, and aggregation. Data node operations are CPU and memory intensive and require sufficient resources to handle the data load. The maximum recommended heap size for any Elasticsearch node is 30GB. It is recommended to scale horizontally rather than vertically to increase the capacity of the cluster.

[0164] (4) Client Nodes: Client nodes act as a gateway to the cluster, helping to balance incoming ingestion and search requests. This is particularly useful in search use cases. Each search has an extraction and reduction phase. During the reduction phase, results from various shards need to be put together. If a data node is under pressure and also acts as a client node, it will not only affect the query results of the shards residing on that node, but also the queries coordinated through that node.

[0165] (5) Ingest Nodes: Ingest nodes can be configured to pre-process data before ingestion. Since some processors (such as grok processors) may consume more resources, it is beneficial to allocate independent nodes for the ingestion pipeline so that search operations are not affected by ingestion processing.

[0166] 2. Word segmentation technology

[0167] Word segmentation is to split text data into searchable terms, establish an inverted index to achieve fast data retrieval, and improve search accuracy. It optimizes the database search experience, improves data retrieval efficiency, and provides a structured foundation for text data analysis and full-text search.

[0168] like Figure 4 As shown, the present invention adopts a statistical word segmentation method - Hidden Markov Model (HMM) word segmentation method, and the basic principle is as follows:

[0169] (1) The principle of word segmentation technology based on HMM (Hidden Markov Model) is to regard the word segmentation task as a sequence labeling problem. In HMM, each character is regarded as being in a specific state, which represents the beginning (B), middle (M), end (E) or a separate word (S) of a word. The model estimates the transition probability between states and the probability of observing a specific Chinese character in each state by training a large amount of annotated corpus data.

[0170] (2) HMM needs to determine three probability matrices: the initial state probability matrix, the state transition probability matrix, and the output observation probability matrix. The initial state probability matrix describes the probability distribution of each state at the beginning of the sequence; the state transition probability matrix describes the probability of transitioning from one state to another; and the output observation probability matrix describes the probability of observing a specific word in a given state.

[0171] (3) During the word segmentation process, for a given sentence, HMM will try to find the most likely state sequence, that is, the most likely word segmentation result. This is usually achieved through the Viterbi Algorithm, which is a dynamic programming algorithm that can retain the local optimal path during the calculation process and ultimately obtain the globally optimal word segmentation result.

[0172] (III) Inverted index technology

[0173] The inverted index is used to store the mapping of the storage location of a word in a document or a group of documents under full-text search. It is a data structure in database retrieval and is stored in the documents of each node of the cluster. The inverted index allows fast full-text search by associating each word in the document with a list of documents containing the word, so that the query can directly locate the document containing a specific vocabulary without scanning the entire document collection. The inverted index mainly consists of two parts:

[0174] (1) Word dictionary: A collection of terms. A term is a separate word or value in a document field generated after word segmentation. The word dictionary records the words of all documents and generally has a large amount of data. It also records the association information (document ID) from the word to the inverted list.

[0175] (2) Inverted list: It consists of inverted index items, mainly including the following contents: ① Document ID, used to obtain original information. ② Word frequency (TF), records the number of times a word appears in the document, used for subsequent relevance algorithms. ③ Position, records the word segmentation position in the document, used for word indexing. ④ Offset, records the start and end positions of the word in the document, used for highlighting.

[0176] To create an inverted index, we split the content of each document into individual terms through word segmentation, create a sorted list of all unique terms, and then list the documents in which each term appears. For example:

[0177] Table 1 Document Contents:

[0178] Table 2 Inverted list:

[0179] Multi-time series data search method

[0180] As the equipment in the communication room becomes increasingly complex, the scale of time series data generated in the room has exploded, including various types of time series data in communication equipment indicators, network data, and sensor data, which are continuously collected at fixed or irregular time intervals. The structure of time series data is not complex, and its typical characteristics are high frequency of generation and large amount of data. Massive time series data puts higher requirements on storage and query capabilities.

[0181] The present invention mainly adopts a fast indexing method based on an improved B-tree.

[0182] When the system has many nodes, the commonly used search algorithm will waste more search time on the communication between nodes, which not only leads to a serious decrease in search efficiency but also affects the network stability of the system. Therefore, when the system has many nodes, a search algorithm with better search efficiency and less network communication resource occupation should be used to ensure the real-time and stability of the simulation system.

[0183] The improved B-tree search algorithm is as follows:

[0184] Usually, the management objects in the system have type attributes, such as power equipment, network equipment, security equipment, etc. Moreover, most of the search nodes in the system are limited to the same belonging group, so if each object is given a different color to express such attribute values, the search range can be narrowed to the nodes with the same color keywords, thereby reducing the search order of the B search tree, accelerating the search efficiency of the keywords inside the B search tree nodes, and reducing unnecessary searches.

[0185] This time we will use the example of two color attributes, and the B-search tree with only two colors is called a red-blue B-tree. (There can be multiple color attributes here, but if there is only one, it will degenerate into a general B-tree.)

[0186] Here, the rules of the m-order red-blue B-tree are given with reference to the rules of the B-tree. n is the total number of nodes, and m is the order of the red-blue B-tree.

[0187] 1. The root node can be a single leaf node;

[0188] 2. The root node should have at least 2 or more child nodes;

[0189] 3. The number of child nodes of all member nodes in the red-blue B-tree should not exceed the order;

[0190] 4. Non-leaf nodes should have at least m / 2 whole children;

[0191] 5. The keywords in non-leaf nodes are ordered sequences;

[0192] 6. All leaf nodes are located in the same layer and can be replaced by NULL.

[0193] 7. All nodes have red or blue attributes;

[0194] 8. All keywords should be linked to the next keyword of the same color.

[0195] like Figure 5 As shown, according to the definition rules of the red-blue B-tree above, when inserting a member node in the tree, you cannot directly add a leaf node, otherwise it will violate its definition rules. Here is the process of inserting a member node in the red-blue B-tree:

[0196] 1. Add a keyword to the corresponding non-leaf node at the lowest level. If the number of member keywords does not exceed m-1, the insertion is completed, otherwise proceed to the second step;

[0197] 2. Split the member nodes according to the splitting rules until they meet the defined rules.

[0198] 3. Create a new index for the keywords of the same-color keywords of related nodes, that is, the keywords of the previous keyword of the same color should contain the keyword information of the next keyword of the same color.

[0199] The splitting rule of the red-blue B-tree member node is: divide the keyword into two parts from the middle of the member node, the left one is smaller than the middle one, and the right one is larger than the middle one. The left keyword is placed in the original node, the right keyword is placed in the new node, and the middle keyword and the new node are inserted into the parent node. If the parent node also has more than m-1 keywords, it needs to be split again until all nodes meet the rules.

[0200] According to the above rules, the process of constructing a red-blue B-tree of order 3 is given here. The seven nodes with keywords 1-7, 1, 3, 5 color attributes are red, 2, 4, 6 color attributes are blue. The green key value represents the key value of the next keyword of the same color in the same node. Red: node key value of red attribute. Blue: node key value of blue attribute.

[0201] From the definition and construction process of the red-blue B-tree, we can know that its search process starts from the root node and searches for the keyword sequence in the node. If it can be found, it returns to the end, otherwise it needs to search in the tree nodes in the range of the keyword; repeat the recursion in sequence until the corresponding keyword is found or the leaf node is found, then the search ends.

[0202] The keywords in the tree nodes of the red and blue B-tree are all arranged in order, and the keywords of the same color are also arranged in order. Therefore, you can use sequential search or binary search to find keywords within the same color range. Here is the arrangement and search process of keywords in the node as shown in the following figure: if the color after the keyword is the same, the index is 0, otherwise it is the keyword information of the next keyword of the same color. For example, after the 2 keyword node is a blue node, that is, the index value of the next red node 7 is set after the 2 keyword. Assuming that you want to find the red 17 keyword, you only need to search the 6 keywords 2, 7, 17, 19, 21, 22 sequentially through the binary search. If you are looking for the minimum belonging interval, after finding the minimum interval of the red keyword, introduce the blue node to further refine the interval. The search result is a successful comparison or finding the minimum belonging interval.

[0203] Test results and analysis

[0204] The present invention adopts ES database and improved B-tree fast indexing method. By improving the traditional B-tree indexing method, the red and blue B-tree successfully combines the low traffic of B-tree and the fast query of keywords. to In terms of time complexity, although the time complexity of searching nodes in red and blue B-tree is the same as that of B-tree, However, the color separation strategy of the red and blue B-tree actually reduces the search range of key nodes to a certain extent, thus obtaining a smaller search length, thereby ensuring higher search efficiency. This method is suitable for systems with a large number of nodes, which can ensure good search efficiency and network utilization at the same time.

[0205] To compare the search efficiency of the search algorithms, Table 1 gives the comparative data of the average search length and time complexity of the balanced binary tree, the 4th-order B-tree and the 4th-order red-blue B-tree when there are 10,100 nodes.

[0206] Table 3 Comparison of average search length and time complexity:

[0207] Through this method, the keywords are accurately located, which greatly improves the search and search speed; saves the search time on the node, improves the search efficiency and the network stability of the system; adopts a search algorithm with better search efficiency and less network communication resource occupation to ensure the simulation real-time and stability of the simulation system; most of the search nodes in the system are confined to the same belonging group, so if a different color is given to each object to express such attribute values, the search range can be narrowed to the nodes with the same color keywords, thereby reducing the search order of the B search tree, accelerating the search efficiency of the keywords inside the B search tree nodes, and reducing unnecessary searches.

[0208] In some specific embodiments, Figure 7 As shown, a fast indexing device based on an improved B-tree search algorithm includes at least one processor and a memory storing instructions to implement any one of the improved B-tree construction methods in the above technical solutions.

[0209] In some specific embodiments, a computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the above technical solution.

[0210] In some specific embodiments, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the method of implementing the steps in the above technical solution is implemented.

[0211] The embodiments and functional operations of the subject matter described in this specification may be implemented in: digital electronic circuitry, tangibly implemented computer software or firmware, computer hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of more than one of the above. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on one or more tangible non-transitory program carriers, for execution by a data processing device or to control the operation of a data processing device.

[0212] Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to an appropriate receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of the above devices.

[0213] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script or code) may be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and the computer program may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a document in a file system. A program may be stored in a portion of a document that stores other programs or data, for example, one or more scripts stored in the following: in a markup language document; in a single document dedicated to a related program; or in multiple collaborative documents, for example, documents storing one or more modules, subroutines or code portions. A computer program may be deployed to execute on one or more computers, the computers being located in one place or distributed to multiple locations and interconnected by a communications network.

[0214] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0215] Although the present invention has been described in detail above by general description, specific implementation methods and experiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A fast search method for a database, comprising a fast indexing method based on an improved B-tree search algorithm, characterized in that: It also includes a non-time series data search method, and the method server includes the following steps: Receive keyword information from the client; Processing the keyword information into a single marked word segmentation element through lexical analysis; Using an inverted index algorithm, matching the stored documents including the keyword in the constructed inverted index; Sending the stored document to the client; The fast indexing method based on the improved B-tree search algorithm comprises the following steps: Assign color attributes according to the keyword type attribute of each node in any tree data structure; Establishing an index linking the current node to the next node with the same color keyword, the index includes an index value, and the index value is a keyword or empty; When the next node with the same color keyword adjacent to the current node includes a node with a different color keyword, the keyword of the current node is the same as the index value of the next node with the same color keyword; When the next node with the same color keyword adjacent to the current node does not include the node with a different color keyword, the index value of the next node with the same color keyword is empty.

2. A method for rapid database search according to claim 1, characterized in that: The current node is a non-leaf node at the bottom of the tree data structure, and the fast indexing method further includes the following steps of creating a new non-leaf node: When a new keyword needs to be added, first create a new non-leaf node at the bottom layer of the non-leaf node of the tree data structure and adjacent to the right side of the current node, then insert the new keyword into the new non-leaf node, and the current node and the new non-leaf node form a member node; If the sum of the number of keywords in the member nodes is less than or equal to m-1, the new non-leaf node is inserted, where m is the order; An index linking the new non-leaf node to the previous non-leaf node with the same color keyword is established.

3. A method for rapid database search according to claim 2, characterized in that: The member node also includes a left adjacent node adjacent to the left side of the current node, the keyword of the member node includes a left keyword, a middle keyword and a right keyword, and the step of creating a new non-leaf node also includes: If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

4. A method for rapid database search according to claim 3, characterized in that: The step of creating a new non-leaf node also includes: If the number of keywords of the previous level member node is greater than m-1, repeat the following steps: If the sum of the number of keywords in the member nodes is less than or equal to m-1, the insertion of the new non-leaf node is completed; If the sum of the number of keywords in the member nodes is greater than m-1, the left keyword is inserted into the left adjacent node, the right keyword is inserted into the new non-leaf node, the middle keyword is inserted into the current node, and the current node with the middle keyword inserted is inserted into the upper-level member node.

5. A method for rapid database search according to claim 1, characterized in that: The nodes of the tree data structure having the same color keywords form a plurality of ordered sequences of the same color keywords, and the fast indexing method further comprises the following keyword query step: Selecting an ordered sequence of keywords of the same color that have the same color attribute as the keyword to be queried, and querying each node having the keyword of the same color; If the index value of the current node with the same color keyword is empty, query the next adjacent node with the same color keyword; If the index value of the current node with the same color keyword is the keyword, jump to the next node with the same color keyword corresponding to the keyword; Until a node with the same color keyword as the keyword to be queried is found.

6. A method for rapid database search according to claim 5, characterized in that: The keywords of all the nodes of the tree data structure form a mixed color ordered sequence; the fast indexing method also includes the following keyword query step: Selecting the ordered sequence of same-color keywords with different color attributes from the keyword to be queried, and finding a minimum interval in the ordered sequence of same-color keywords with different color attributes; the minimum interval is two consecutive nodes on the ordered sequence of same-color keywords with different color attributes, and a section of mixed-color ordered sequence intercepted from the mixed-color ordered sequence based on the two consecutive nodes, the section of mixed-color ordered sequence including the keyword to be queried; The intercepted section of the mixed color ordered sequence is inserted between the two continuous nodes, and then the keyword to be queried is searched in the intercepted section of the mixed color ordered sequence.

7. A method for rapid database search according to claim 6, characterized in that: The nodes with the same color keywords in the tree data structure form two ordered sequences of the same color keywords.

8. A fast search device for a database, the device comprising at least one processor and a memory storing instructions, characterized in that: When the instruction is executed by at least one processor, a method for quickly searching a database as described in any one of claims 1 to 7 is implemented.

Images