Method for constructing a word dictionary based on a double array tree and a word query method

Abstract

The application provides a double-array tree-based dictionary construction method and word query method, which comprises the following steps: generating a sensitive word tree based on the split text characters of each sensitive word and the arrangement order of the text characters, wherein each path in the sensitive word tree corresponds to a sensitive word; performing dictionary coding on the split text characters of the sensitive word after deduplication to obtain the character coding corresponding to each text character; determining the base coding of each character node in the sensitive word tree in layers according to at least one of random selection trial calculation, self-increment selection trial calculation, overall query selection trial calculation and preset data selection trial calculation; determining the storage location coding of the text character of each character node in the storage space based on the character coding corresponding to the text character of each character node and the base coding of the parent node of each character node, and taking the storage location coding of the parent node of each character node as the check coding of each character node, so as to construct a sensitive word dictionary.

Description

Technical Field

[0001] This application relates to the field of sensitive word recognition technology, and in particular to a method for constructing a word library and a word query method based on a double-array tree. Background Technology

[0002] In existing technologies, all sensitive words are typically stored in their full text in a sensitive word dictionary. However, this full-text storage method results in a large amount of duplicate data; for example, the sensitive words "AB" and "AC" will both contain the duplicate "A". Furthermore, full-text storage requires full-text matching during queries, leading to a large workload and long query time. Alternatively, all sensitive words can be constructed into a sensitive word tree for storage to overcome the duplicate storage problem. However, tree structures require storage in a structured format, and the contextual relationships between parent and child nodes require extensive structural code. This results in discontinuous physical address storage of nodes during storage, requiring significant memory for the sensitive word tree, and necessitates frequent pointer jumps during queries, leading to long query times. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide at least one method for constructing a word library and a word query method based on a double-array tree. By reorganizing the sensitive word library through a double-array tree, the technical problems of large storage space and long sensitive word recognition time in the prior art are solved, thereby achieving the technical effect of reducing the storage space of the sensitive word tree and reducing the sensitive word recognition time.

[0004] This application mainly includes the following aspects:

[0005] In a first aspect, embodiments of this application provide a method for constructing a lexicon based on a dual-array tree. The method includes: generating a sensitive word tree based on the text characters split from each sensitive word and the arrangement order of each text character, wherein each path in the sensitive word tree corresponds to a sensitive word; deduplicating the text characters split from the sensitive word and performing dictionary encoding to obtain character codes corresponding to each text character; determining the basic code of each character node in the sensitive word tree in a hierarchical manner according to at least one of random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation; determining the storage location code of the text character of each character node in the storage space based on the character code corresponding to the text character of each character node and the basic code of its parent node, and using the storage location code of the parent node of the character node as the check code of the character node to construct the sensitive word lexicon.

[0006] Secondly, embodiments of this application also provide a word query method based on a dual-array tree. The method utilizes the sensitive word library created by the method described in the first aspect or any possible implementation of the first aspect. The method includes: determining whether a first query character in the sensitive word to be queried is a dictionary-encoded text character; if so, determining a first target storage location encoding of the first query character in the storage space based on the preset basic encoding of the root node of the sensitive word library and the character encoding corresponding to the first query character of the sensitive word to be queried; and determining whether the text character stored at the first target storage location is the first query character, and the first target storage location, based on the first target storage location encoding. If the check code corresponding to the set code is the storage location code of the root node, then determine whether the base code corresponding to the first target storage location code is the preset end base code; if not, based on the base code corresponding to the first target storage location code and the character code corresponding to the second query character of the sensitive word to be searched, determine the second target storage location code of the second query character in the storage space; based on the text character stored in the storage location corresponding to the current target storage location code, the base code and the check code, determine whether the current query character meets the sensitive word character determination condition; if all conditions are met, then determine the sensitive word to be searched as a sensitive word stored in the sensitive word library.

[0007] Thirdly, this application also provides a vocabulary building device based on a dual-array tree. The device includes: a sensitive word tree generation module, which generates a sensitive word tree based on the text characters split from each sensitive word and the arrangement order of each text character, wherein each path in the sensitive word tree corresponds to a sensitive word; an encoding module, which performs dictionary encoding on the text characters split from the sensitive word after deduplication to obtain the character encoding corresponding to each text character; a basic encoding determination module, which determines the basic encoding of each character node in the sensitive word tree in a hierarchical manner according to at least one of random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation and preset data selection trial calculation; and a construction module, which determines the storage location encoding of the text character of each character node in the storage space based on the character encoding corresponding to the text character of each character node and the basic encoding of its parent node, and uses the storage location encoding of the parent node of the character node as the check encoding of the character node to construct the sensitive word library.

[0008] Fourthly, embodiments of this application also provide a word query device based on a dual-array tree. The device utilizes the sensitive word library created by the method described in the first aspect or any possible implementation of the first aspect. The device includes: a first determining module, which determines whether a first query character in the sensitive word to be queried is a dictionary-encoded text character; a second determining module, if so, which determines, based on the preset basic encoding of the root node of the sensitive word library and the character encoding corresponding to the first query character of the sensitive word to be queried, a first target storage location encoding of the first query character in the storage space; and a checking module, which determines, based on the first target storage location encoding, whether the text character stored at the storage location is the first query character, and the first target storage location encoding. The following steps are taken: 1) Check whether the corresponding check code is the storage location code of the root node; 2) Determine whether the base code corresponding to the first target storage location code is the preset end base code; 3) Determine whether the base code corresponding to the first target storage location code is the second target storage location code of the second query character in the storage space, based on the base code corresponding to the first target storage location code and the character code corresponding to the second query character of the sensitive word to be searched; 4) Determine whether the current query character meets the sensitive word character determination conditions, based on the text character stored in the storage location corresponding to the current target storage location code, the base code, and the check code; 5) If all conditions are met, determine the sensitive word to be searched as a sensitive word stored in the sensitive word database.

[0009] This application provides a method for constructing a word library and querying words based on a dual-array tree. The method includes: generating a sensitive word tree based on the text characters split from each sensitive word and the arrangement order of each text character, wherein each path in the sensitive word tree corresponds to a sensitive word; deduplicating the text characters split from the sensitive word and performing dictionary encoding to obtain the character encoding corresponding to each text character; determining the basic encoding of each character node in the sensitive word tree in a hierarchical manner according to at least one of random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation; determining the storage location encoding of the text character of each character node in the storage space based on the character encoding corresponding to the text character of each character node and the basic encoding of its parent node, and using the storage location encoding of the parent node of the character node as the check encoding of the character node to construct the sensitive word library. By splitting sensitive words and generating a sensitive word tree according to the arrangement of the text characters of each sensitive word, the deduplication of each text character after splitting is performed and dictionary encoding is performed to obtain the character encoding of all text characters. The basic encoding of each character node in each layer of the sensitive word tree is determined sequentially by at least one of the following methods: random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation. The storage location encoding of the text character of each character node in the sensitive word tree is determined according to the character encoding corresponding to the text character of each character node and the basic encoding of its parent node. The storage location encoding of the parent node of each character node is used as the check encoding of each character node to construct a sensitive word library. This solves the technical problems of large storage space and long sensitive word recognition time in the prior art, and achieves the technical effect of reducing the storage space of the sensitive word tree and reducing the sensitive word recognition time.

[0010] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of a dictionary construction method based on a dual-array tree provided in an embodiment of this application is shown.

[0013] Figure 2 A schematic diagram of the sensitive word tree provided in an embodiment of this application is shown.

[0014] Figure 3 The diagram shows a functional block diagram of a dictionary construction device based on a dual-array tree provided in an embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0016] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0017] In existing technologies, a sensitive word database is constructed by exhaustively searching all sensitive words. This storage method results in a large amount of duplicate data storage. For example, the sensitive words "AB" and "AC" will both store "A", leading to a large storage space for the sensitive word database. Furthermore, sensitive word recognition requires traversing every sensitive word in the database, resulting in a long recognition time. Alternatively, a sensitive word tree can be constructed for all sensitive words, storing different sensitive words by pairing parent nodes with different child nodes, thus overcoming the problem of duplicate storage. However, the constructed sensitive word tree requires storing the connection relationships between different nodes in a structure format, making the storage space between different characters in the sensitive word group discontinuous. For example, a parent node occupies one position in memory, and a child node of the parent node needs to be stored with some structure storage space between them. This results in a large amount of structure code in the memory where the sensitive word tree is stored. When recognizing sensitive words, it is necessary to repeatedly read and compile the structure code, causing the query pointer to jump repeatedly, resulting in slow and inefficient sensitive word recognition.

[0018] Based on this, this application provides a method for constructing a word library and a word query method based on a dual-array tree. It generates a sensitive word tree by splitting sensitive words and arranging the text characters of each sensitive word in order. The split text characters are deduplicated and dictionary-encoded to obtain the character codes of all text characters. The basic code of each character node in each layer of the sensitive word tree is determined sequentially using at least one of random selection trial calculation, auto-incrementing selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation. The storage location code of each character node's text character is determined according to the character code corresponding to each character character in the sensitive word tree and the basic code of its parent node. The storage location code of each character node's parent node is used as the check code for each character node to construct a sensitive word library. This solves the technical problems of large storage space and long sensitive word recognition time in the prior art, achieving the technical effect of reducing the storage space of the sensitive word tree and reducing the sensitive word recognition time. Specifically, as follows:

[0019] Please see Figure 1 , Figure 1 This is a flowchart illustrating a dictionary construction method based on a dual-array tree, provided as an embodiment of this application. Figure 1 As shown in the embodiments of this application, the lexicon construction method based on a dual-array tree includes the following steps:

[0020] S101: Generate a sensitive word tree based on the text characters split from each sensitive word and the order of these text characters.

[0021] Each path in the sensitive word tree corresponds to a sensitive word.

[0022] In other words, each sensitive word is split into at least one text character, and the first text character of all sensitive words, after deduplication, is used as the first-level character node of the sensitive word tree. This process is repeated layer by layer according to the order of the text characters in each sensitive word, so that each path in the sensitive word tree points to a sensitive word. Specifically, for each path in the sensitive word tree, the text characters corresponding to the first-level character node to the bottom-level character node of that path are combined to form a sensitive word.

[0023] Furthermore, after splitting and deduplicating all sensitive words, each text character obtained may be a first-level character node, or there may be only one text character that is a first-level character node. That is, the number of first-level character nodes is no greater than the number of deduplicated text characters. Each text character in each sensitive word corresponds to a character node in the sensitive word tree. In the sensitive word tree, there may be cases where the same text character is set in multiple character nodes, or there may be cases where different character nodes correspond to different text characters.

[0024] Please see Figure 2 , Figure 2 This is a schematic diagram of a sensitive word tree provided in an embodiment of this application. Figure 2 As shown, the root node of the entire sensitive word tree is R. If multiple sensitive words include AB, ABB, ABC, AD, BCB, BCD, CBB, CBD, and DAC, each sensitive word is split into multiple text characters. After removing duplicates from the first text character, the first-level character nodes A, B, C, and D of the sensitive word tree are obtained. For the first-level character node A, the sensitive words whose first text character is A are AB, ABB, ABC, and AD. Furthermore, the child nodes of the first-level character node A include child nodes B and D. The first-level character node A and its corresponding child node B can directly form the sensitive word AB. Therefore, the first-level character node A combined with its child node B directly forms a path in the sensitive word tree. The first-level character node A and its corresponding child node B also need to be combined with its grandchild node B to form the sensitive word ABB. The first-level character node A and its corresponding child node B also need to be combined with its grandchild node C to form the sensitive word ABC. The first-level character node A and its corresponding child node D can directly form the sensitive word AD. Therefore, the first-level character node A combined with its child node D directly forms a path in the sensitive word tree. The same logic applies to the first-level character nodes B, C, and D, which will not be elaborated further here. This is how the sensitive word tree is constructed.

[0025] In other words, a sensitive word tree includes at least a root node and character nodes. Character nodes correspond to the text characters of sensitive words. The root node of the sensitive word tree only serves as the parent node of all first-level character nodes and does not correspond to any text characters.

[0026] For example, multiple sensitive words refer to words that need to be identified and processed in advance, such as words that need to be blocked. Furthermore, a text character can correspond to any of the following: a Chinese character, a symbol, a number, a letter, and a word. This application does not limit this.

[0027] S102: After deduplication of the text characters split from the sensitive words, dictionary encoding is performed to obtain the character codes corresponding to each text character.

[0028] In other words, all sensitive words are broken down into individual text characters, and all text characters are deduplicated. The deduplicated text characters are then dictionary-encoded, with each text character corresponding to a character code. The text characters are identified by their individual character codes.

[0029] For example, please refer to Table 1, which is a table showing the correspondence between each text character and its corresponding character code. If multiple sensitive words include: AB, ABB, ABC, AD, BCB, BCD, CBB, CBD, and DAC, the text characters obtained after splitting each sensitive word into multiple text characters and removing duplicates are A, B, C, and D.

[0030] Table 1:

[0031]

[0032] In other words, each text character is configured with a character encoding. The character encoding can be set by the user, either by setting the character encoding step by step in the order of integers, or by setting the character encoding randomly. As long as the character encoding of each text character is not repeated, this application does not impose any restrictions on this.

[0033] Specifically, the sensitive word tree also includes end leaf nodes for indicating the end of the path, each end leaf node includes an end character, and the method further includes: performing dictionary encoding on the end character to obtain the end character encoding corresponding to the end character.

[0034] In this context, the parent node of the ending leaf node is the lowest-level character node of a path in the sensitive word tree, or in other words, the parent node of the ending leaf node corresponds to the character node of the last text character of the sensitive word. Furthermore, the ending leaf node itself does not correspond to a text character, but its parent node does.

[0035] In this system, each ending leaf node corresponds to an ending character. The ending characters for each ending leaf node can be different, or they can all correspond to the same ending character. The ending character for each ending leaf node should be different from the text character to distinguish whether the path has ended.

[0036] Furthermore, when each ending leaf node corresponds to a different ending character, these ending characters can be set in ascending integer order, with the number of ending characters equal to the total number of sensitive words in the sensitive word tree. This allows for rapid identification of which sensitive word in the sensitive word tree the text to be identified matches. When each ending leaf node corresponds to the same ending character, a single ending character is used to represent the end of each path.

[0037] In other words, the ending characters of the ending leaf nodes associated with different paths in the sensitive word tree can be the same or different. For example, the ending characters of all ending leaf nodes can be set to 0, or different paths can be associated with different ending leaf nodes.

[0038] When performing dictionary encoding on the ending characters of each ending leaf node, if each ending leaf node corresponds to a different ending character, dictionary encoding can either assign the same ending character code to all different ending characters or assign a different ending character code to each ending character. If each ending leaf node corresponds to the same ending character, only one ending character needs to be dictionary encoded, i.e., one ending character code is used to represent the ending character of each ending leaf node. Furthermore, the ending character code should be different from the character code of the text characters to distinguish them.

[0039] like Figure 2 As shown, the ending character of the ending leaf node is different for each path. For the first-level character node A, the first-level character node A paired with its child node B forms a path corresponding to the sensitive word AB, and this path is set with an ending character 1; the first-level character node A, its corresponding child node B, and its grandchild node B form a path corresponding to the sensitive word ABB, and this path is set with an ending character 2; the first-level character node A, its corresponding child node B, and its grandchild node C form a path corresponding to the sensitive word ABC, and this path is set with an ending character 3; the first-level character node A and its other corresponding child node D form a path corresponding to the sensitive word AD, and this path is set with an ending character 4. The ending characters of the paths corresponding to the first-level character nodes B, C, and D are similar and will not be repeated here. Therefore, Figure 2 The ending characters include ending character 1 to ending character 9. Each ending character is dictionary-encoded, and different ending character codes can be set for ending character 1 to ending character 9.

[0040] For example, please refer to Table 2, which is a table showing the correspondence between each ending character and its corresponding ending character code. If the ending characters include ending characters 1 to 9, the ending character code for each ending character is set to a negative number.

[0041] Table 2:

[0042]

[0043] return Figure 1 S103: The basic encoding of each character node in the sensitive word tree is determined in layers according to at least one of the following methods: random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation.

[0044] Among them, random selection trial calculation refers to randomly setting the basic code for each character node; auto-increment selection trial calculation refers to setting the basic code for each character node in ascending order of integers; overall query selection trial calculation refers to storing multiple character nodes as a whole and setting their basic codes; and preset data selection trial calculation refers to selecting from various preset numbers in a preset data set as the basic codes for each character node. Therefore, it is possible that some character nodes share the same basic code, or that each character node has a different basic code.

[0045] For example, the base code of the root node in the sensitive word tree can be set to be unique to the root node only. That is, the base code of the root node can be preset to be different from the base codes of any other character node and the ending leaf node. Alternatively, it can be preset to be the same as the base code of any character node and / or the ending leaf node; this is not limited here. The base code of the ending leaf node in the sensitive word tree can be set in the same way as the base codes of the character nodes; or the base code of the ending leaf node can be set to a preset ending base code unique to the ending leaf node, meaning the preset ending base code is different from the base codes of the root node and any other character node.

[0046] Specifically, the step of determining the basic encoding of each character node in the sensitive word tree in a sequential layered manner includes: setting the virtual encoding of the character node according to a preset trial calculation rule, and determining at least one storage location encoding corresponding to the storage space required by all child nodes of the character node through the virtual encoding and the character encoding of all child nodes of the character node; and using the virtual encoding corresponding to the storage location of each storage location encoding when all storage locations are free as the basic encoding of the character node.

[0047] In other words, for each character node in the sensitive word tree (each character node corresponding to each path in the sensitive word tree), a virtual code for the character node is assumed according to a preset trial calculation rule. The assumed virtual code and the character codes of all child nodes of the character node are used to determine whether the storage location corresponding to at least one storage location code that accommodates all child nodes of the character node is free. When the storage location corresponding to all storage location codes is free, the assumed virtual code is used as the basic code of the character node.

[0048] The preset trial calculation rule can refer to gradually increasing the virtual code starting from a preset minimum virtual code. If the assumed virtual code cannot find a storage location code where all storage locations are free, then the assumed virtual code is increased by a preset value to calculate the virtual code for the next assumption. The preset value can be 1. Alternatively, the preset value used each time can be set to a different value. For example, if the assumed virtual code does not meet the requirements, then one is added to the assumed virtual code as the virtual code for the next assumption. If it still does not meet the requirements, then three is added to the virtual code for the next assumption, and so on, increasing in an odd-number increment order. Alternatively, the virtual code of each character node in the sensitive word tree can be determined by at least one of random selection trial calculation, auto-increment selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation. These methods will not be elaborated further here.

[0049] The storage location encoding refers to the encoding corresponding to the storage location of each node (including the root node, character node, and ending leaf node) in the sensitive word tree. Different storage locations correspond to different storage location encodings; that is, each node in the sensitive word tree corresponds to a unique storage location encoding. Thus, the storage location encoding of each node reflects the storage location of the root node, the text character corresponding to the character node, or the ending character corresponding to the ending leaf node.

[0050] The storage location encoding can also refer to the encoding corresponding to the storage location of the basic encoding and / or check encoding of each node (including the root node, character node, and end leaf node) in the sensitive word tree. Different storage locations correspond to different storage location encodings, that is, the basic encoding and / or check encoding of each node in the sensitive word tree each correspond to a storage location encoding.

[0051] For example, for each first-level character node in the sensitive word tree, the sum of the base code of the parent node (i.e., the root node of the sensitive word tree) and the character code of the first-level character node is used as the storage location code of the first-level character node. The base code of the root node of the sensitive word tree is a preset base code, which can be set to 1. Furthermore, the child nodes of the root node in the sensitive word tree are first-level character nodes, or in other words, the character nodes obtained after deduplicating the first text character of all sensitive words are used as first-level character nodes. Figure 2As shown, the first-level character nodes of the sensitive word tree include first-level character node A, first-level character node B, first-level character node C, and first-level character node D. Furthermore, the storage location encoding of first-level character node A is the sum of the root node's base encoding 1 and the character encoding 1 of character node A; the storage location encoding of first-level character node B is the sum of the root node's base encoding 1 and the character encoding 2 of character node B; the storage location encoding of first-level character node C is the sum of the root node's base encoding 1 and the character encoding 3 of character node C; and the storage location encoding of first-level character node D is the sum of the root node's base encoding 1 and the character encoding 4 of character node D. Please refer to Table 3, which refers to the double-array table of the first-level character nodes.

[0052] Table 3:

[0053]

[0054] In other words, for each character node, by determining whether the storage location corresponding to at least one storage location code corresponding to the storage space required by all child nodes of the character node is free, it is determined whether the sensitive word library can hold all child nodes of the character node. If the storage locations corresponding to the storage location codes corresponding to all child nodes of the character node are free, it means that the storage location of all child nodes of the character node can be found in the sensitive word library through virtual encoding. Thus, the virtual encoding set in this case can be used as the basic encoding of the character node.

[0055] For example, each storage location code can correspond to a fixed space length in the storage space. For instance, each storage location code may cover one byte or multiple bytes, or each storage location code may cover a different number of bytes. This application does not impose any restrictions on this. If each storage location code covers 2 bytes, then the storage space corresponding to each storage location code occupies 16 bits.

[0056] The step of setting the virtual code of the character node according to the preset trial calculation rules includes: determining whether the overall query conditions are met based on the number of child nodes of the character node and the character codes corresponding to each child node; if the overall query conditions are met, determining the value of the virtual code based on the difference between the smallest and largest character codes among the child nodes that meet the overall query conditions.

[0057] Specifically, for each character node, all its child nodes are divided into a first number of partial child nodes and a second number of other child nodes. The first number is at least one, and the second number is the difference between the total number of all child nodes and the number of partial child nodes. Furthermore, the overall query condition refers to treating a partial child node as a whole to determine whether the storage space of the sensitive word database can accommodate that partial child node. If the first number of partial child nodes equals the total number of all child nodes of the character node, and the second number is 0, then all child nodes of the character node are considered as a whole. If the first number of partial child nodes is less than the total number of all child nodes of the character node, and the second number is not 0, then the partial child nodes of the character node are considered as a whole.

[0058] Furthermore, if the first number of some child nodes is equal to the total number of all child nodes of the character node, then the child nodes that satisfy the overall query conditions refer to all child nodes of the character node. Furthermore, the difference between the smallest and largest character codes among the child nodes that satisfy the overall query conditions refers to the difference between the smallest and largest character codes among all child nodes of the character node. If the first number of some child nodes is less than the total number of all child nodes of the character node, then the child nodes that satisfy the overall query conditions refer to some child nodes of the character node. Furthermore, the difference between the smallest and largest character codes among the child nodes that satisfy the overall query conditions refers to the difference between the smallest and largest character codes among some child nodes of the character node.

[0059] The determination of whether the overall query conditions are met, based on the number of child nodes of the character node and the character codes corresponding to each child node, includes: taking the difference between the largest and smallest character codes of all child nodes of the character node as the maximum difference of all child nodes corresponding to the character node; determining whether the number of all child nodes of the character node is greater than a first preset value, and determining whether the ratio of the maximum difference of all child nodes corresponding to the character node to the number of all child nodes is less than a second preset value; when the number of all child nodes of the character node is greater than the first preset value, and the ratio of the maximum difference of all child nodes corresponding to the character node to the number of all child nodes is less than the second preset value, it is determined that all child nodes of the character node meet the overall query conditions.

[0060] Alternatively, determining whether the overall query conditions are met based on the number of child nodes of the character node and the character codes corresponding to each child node includes: taking the difference between the largest and smallest character codes of some child nodes of the character node as the maximum difference of some child nodes corresponding to the character node; determining whether the number of some child nodes of the character node is greater than a first preset value, and determining whether the ratio of the maximum difference of some child nodes corresponding to the character node to the number of some child nodes is less than a second preset value; when the number of some child nodes of the character node is greater than the first preset value, and the ratio of the maximum difference of some child nodes corresponding to the character node to the number of some child nodes is less than the second preset value, it is determined that some child nodes of the character node meet the overall query conditions.

[0061] Furthermore, by using a first preset value and a second preset value, we can measure whether all or some child nodes are numerous and densely arranged. When the overall query conditions are met, it is considered that all or some child nodes are numerous and densely arranged, thus allowing all or some child nodes to be judged as a whole. This is equivalent to treating all or some child nodes of the character node as a whole to search for storage space, which can significantly improve storage efficiency compared to searching for storage space for each individual child node.

[0062] For example, the first preset value is set to 1, and the second preset value is set to 2. Figure 2 As shown, the child nodes of the first-level character node A include the second-level character node B and the second-level character node D. Both characters are greater than the first preset value. The character code of character node B is 2, and the character code of character node D is 4. Then the maximum difference of all child nodes corresponding to the first-level character node A is 4-2=2. The ratio of the maximum difference (Code Range) to the number of all child nodes is 1. The ratio is less than the second preset value. Therefore, it is considered that the two child nodes of the first-level character node A meet the overall query condition of a large number of densely arranged child nodes. Thus, the second-level character nodes B and D are treated as a whole to determine whether the required storage space is free.

[0063] Specifically, when the overall query conditions are met, the value of the virtual code is determined based on the difference between the smallest and largest character codes among the child nodes that meet the overall query conditions. This includes: if all child nodes of the character node meet the overall query conditions, the storage location code corresponding to the smallest character code among the child nodes is used as the starting point of the storage location code corresponding to the storage space required by all child nodes of the character node; the storage space from the storage location code corresponding to the smallest character code among all child nodes to the storage location code corresponding to the largest character code is used as a whole to determine whether it is free; if so, the current virtual code corresponding to the storage location code corresponding to each storage location code is free is used as the basic code of the character node; if not, the value of the virtual code in the next round is set to be at least increased by the difference between the largest and smallest character codes among all child nodes plus 1 on the current virtual code.

[0064] Specifically, all or some of the child nodes of the character node are treated as the entire child node. The storage location corresponding to the storage location code of the entire child node of the character node is determined to be free in the following way: the virtual code of the character node is set as the preset value for the first calculation; the first sum of the virtual code of the character node and the smallest character code of the entire child node is calculated, and the first sum is used as the starting point of the storage location code corresponding to the smallest character code of the entire child node; the second sum is obtained by adding one to the maximum difference (Code Range) of the entire child node of the character node, and the second sum is used as the offset length (Jump size), or the second sum refers to the difference between the smallest and largest character codes among the child nodes that satisfy the overall query conditions plus 1; the length involved from the starting point to the ending point of the storage location code is the maximum difference, and the ending point of the storage location code is determined in this way; when the entire storage location from the starting point to the ending point of the storage location code is free, it is determined that the storage location corresponding to the storage location code of the entire child node of the character node is free.

[0065] For example, when not all storage locations from the start to the end of the storage location encoding are free, the virtual encoding of the previously assumed character node can be increased by the offset length (Jump size) to obtain the current virtual encoding of the character node. Then, the sum of the virtual encoding of the character node and the minimum character encoding of all child nodes can be used as the starting point of the current storage location encoding. Alternatively, when not all storage locations from the start to the end of the storage location encoding are free, the virtual encoding of the previously assumed character node can be increased by twice the offset length (Jump size) to obtain the current virtual encoding of the character node. Then, the virtual encoding of the current character node can be increased by the minimum character encoding of all child nodes to obtain the starting point of the current storage location encoding, thereby improving the efficiency of finding storage locations.

[0066] For example, such as Figure 2 As shown, the child nodes of the first-level character node A include the second-level character node B and the second-level character node D. When determining that the two child nodes of the first-level character node A satisfy the overall query condition of a large number of densely arranged child nodes, it is assumed that the initial virtual code of the first-level character node A is 1. The maximum difference between the virtual code 1 of the first-level character node A and the character code of the second-level character node B with the smallest character code is calculated. The maximum difference between the character codes of all child nodes of the first-level character node A is 2, that is, the difference between the character code 4 of the second-level character node D and the character code 2 of the second-level character node B is 2. The maximum difference 2 is calculated and one is added to obtain the second sum value 3. Therefore, the offset length is 3. Then, the storage location code 3 can be used as the starting point, and the storage location code 5 can be determined according to the maximum difference 2 as the ending point. In this way, at least one storage location code corresponding to all child nodes of the first-level character node A is determined to be storage location code 3 to storage location code 5. It is then determined whether the overall storage locations corresponding to storage location codes 3 to storage location code 5 are all free. As shown in Table 3, storage location code 3 is occupied by the first-level character node B, storage location code 4 is occupied by the first-level character node C, and storage location code 5 is occupied by the first-level character node D. Therefore, the basic code of the first-level character node A cannot be the current virtual code 1.

[0067] For example, when it is determined that the base code of the first-level character node A cannot be the current virtual code 1, the virtual code 1 of the first-level character node A set in the previous step is added to the calculated offset length 3 (i.e., the difference between the maximum and minimum character codes of the child nodes plus 1) as the virtual code of the first-level character node A in this assumption, i.e., 1+3=4. The virtual code of the first-level character node A is then added to the smallest character code among the child nodes (4+2=6) as the starting point of the storage location code for this query. Thus, it is determined whether the overall storage locations corresponding to storage location codes 6 to 8 are all free. If it is found that all storage locations are free, then the base code of the first-level character node A is determined to be the virtual code 4 in this assumption. This avoids trying virtual codes one by one starting from 1, 2, 3, 4, 5, ... when determining the base code of the first-level character node A, and also avoids querying the storage space corresponding to each child node for free after each assumption of a virtual code, greatly improving storage efficiency.

[0068] Furthermore, when the basic encoding of the first-level character node A is determined to be 4, the storage location encoding of the second-level character node B is calculated as the sum of the basic encoding of the parent node of the second-level character node B and the character encoding of the second-level character node B (i.e., 4+2=6), and the storage location encoding of the second-level character node B is 6. The storage location encoding of the second-level character node D is calculated as the sum of the basic encoding of the parent node of the second-level character node D and the character encoding of the second-level character node D (i.e., 4+4=8), and the storage location encoding of the second-level character node D is 8.

[0069] For example, the base encoding can be understood as an offset. That is, the base encoding of the first-level character node A is 4. This can be understood as moving four storage spaces from the storage location encoding corresponding to the character encoding of the second-level character nodes B and D, which are child nodes, to obtain the storage space of the child nodes of the first-level character node A.

[0070] Please refer to Table 4, which is a double array table of the first-level character nodes and the child nodes of the first-level character node A.

[0071] Table 4:

[0072]

[0073] Alternatively, when the overall query conditions are met, the value of the virtual code is determined based on the difference between the smallest and largest character codes among the child nodes that meet the overall query conditions. This includes: if some child nodes of the character node meet the overall query conditions, the storage location code corresponding to the smallest character code among the child nodes that meet the overall query conditions is used as the starting point of the storage location code corresponding to the storage space required by some child nodes of the character node; the storage space from the storage location code corresponding to the smallest character code among the child nodes that meet the overall query conditions to the storage location code corresponding to the largest character code is used as a whole to determine whether it is free; and based on the current virtual code and the character codes of the child nodes of the character node that do not meet the overall query conditions, the storage location codes corresponding to the storage space required by the other child nodes of the character node are determined, and it is determined whether the storage location corresponding to each storage location code is free; if yes, the current virtual code corresponding to the storage location code corresponding to each storage location code is free is used as the basic code of the character node; if no, the value of the virtual code in the next round is set to be at least increased by 1 by the difference between the largest and smallest character codes among the child nodes that meet the overall query conditions.

[0074] In other words, only a subset of the child nodes of the character node are considered as a whole. Assuming the initial virtual code of the character node is 1, the virtual code 1 of the character node is added to the minimum character code of the subset of child nodes to form the storage location code of the child node with the minimum character code. This storage location code is used as the starting point, and the difference between the maximum and minimum character codes of the subset of child nodes, plus 1, is used as the offset length. This determines the endpoint of the storage location code corresponding to the subset of child nodes covering the difference, starting from the endpoint of the storage location code, and whether the overall storage location corresponding to the starting and ending storage location codes of the subset of child nodes is free. For all child nodes of the character node other than the subset of child nodes, for each other child node, the virtual code 1 of the character node is added to the character code of the other child node to form its storage location code, and whether the storage location corresponding to each storage location code of the other child node is free. Only when the storage locations corresponding to the storage location codes of the subset of child nodes and other child nodes are all free is the virtual code 1 of the character node used as the base code 1 of the character node. If the overall storage space corresponding to the storage location encoding of some child nodes is not free, or the storage location corresponding to the storage location encoding of any other child node is not free, then the virtual encoding of the character node is added to the offset length of some child nodes. That is, the virtual encoding of the character node is 1 plus the difference between the maximum and minimum character encoding of some child nodes plus 1, which is used as the virtual encoding of the character node in the next trial calculation.

[0075] Specifically, the method further includes: when the overall query conditions are not met, determining the storage location code corresponding to the storage space required by each child node based on the current virtual code and the character code of each child node of the character node, and determining whether the storage location corresponding to each storage location code is free; if so, using the current virtual code corresponding to the storage location code when all storage locations are free as the basic code of the character node; if not, setting the value of the next round of virtual code to be the current virtual code plus a preset value, or randomly obtaining the value of the next round of virtual code within the preset value range without repetition, or obtaining the value of the next round of virtual code according to the preset value sequence.

[0076] In other words, when the total number of child nodes of a character node is less than or equal to a first preset value, and / or when the ratio of the maximum difference in character encoding of the total child nodes of the character node to the total number of child nodes is greater than or equal to a second preset value, it is determined that the total child nodes of the character node do not meet the overall query conditions. Here, the total child nodes include all child nodes or some child nodes.

[0077] Therefore, if the overall query conditions are not met, it is assumed that the number of all or some child nodes is small and / or their arrangement is not dense, making it impossible to judge all or some child nodes as a whole. Consequently, it is necessary to search for storage space for each of the child nodes of the character node individually.

[0078] In other words, when the overall query conditions are not met, for each character node, a virtual code is assumed, and the sum of this virtual code and the character codes of each of its child nodes is used as the storage location code corresponding to the storage space required by each child node. It is then determined whether the storage location corresponding to each storage location code is free. If all storage locations corresponding to each storage location code are free, the virtual code of the character node is used as its base code. If any storage location corresponding to a storage location code is not free, the value of the next virtual code is set to the sum of the current virtual code and a preset value, or a non-repeating preset value is randomly selected within the preset value range as the value of the next round of virtual codes, or the preset value corresponding to the next sequence of the sequence corresponding to the current round of virtual codes is used as the value of the next round of virtual codes. Furthermore, for each character node, the virtual code is set to a different value for each trial calculation.

[0079] For example, if the first preset value is 1, the second preset value is 2, the child node of the first-level character node B is the second-level character node C, and the number of child nodes of the first-level character node B is not greater than the first preset value 1, the first-level character node B does not meet the overall query condition of having a large number of densely arranged child nodes. The availability of the storage location corresponding to the storage location code of the second-level character node C can be determined by successively incrementing the virtual code of the first-level character node B by 1. Alternatively, the availability of the storage location corresponding to the storage location code of the second-level character node C can be determined by selecting different values ​​of the virtual code of the first-level character node B each time. If the virtual code of the first-level character node B is successively incremented by 1, and assuming the initial virtual code of the first-level character node B is 1, then the storage location code of the second-level character node C is the sum of the virtual code 1 of the first-level character node B and the character code 3 of the second-level character node C, which is 4. At this point, it is necessary to determine whether the storage space with storage location code 4 is free. As can be seen from Table 4, storage location code 4 is occupied by the first-level character node C. Therefore, the virtual code of the first-level character node B needs to be incremented by 1 to reset it to 2. The storage location code of the second-level character node C is calculated again as the sum of the virtual code 2 of the first-level character node B and the character code 3 of the second-level character node C, which is 5. Table 4 shows that storage location code 5 is occupied by the first-level character node D. Therefore, the virtual code of the first-level character node B needs to be incremented by 1 to reset to 3, and so on, until the virtual code of the first-level character node B is set to 4. At this point, the storage location code of the second-level character node C is the sum of the virtual code 4 of the first-level character node B and the character code 3 of the second-level character node C, which is 7. Table 4 shows that the storage location corresponding to storage location code 7 is not occupied. Therefore, the basic code of the first-level character node B can be set to 4, and the storage location code of the second-level character node C is 7. Furthermore, when the number of child nodes is relatively small and / or sparsely arranged, by determining whether the storage location code of each child node is occupied, child nodes can be inserted into free storage locations, improving the utilization of fragmented space, increasing storage utilization, and reducing storage volume.

[0080] Therefore, following the above method, it is possible to Figure 2 The basic encoding of each character node in the sensitive word tree is determined.

[0081] return Figure 1 S104: Based on the character encoding corresponding to the text character of each character node and the basic encoding of its parent node, determine the storage location encoding of the text character of the character node in the storage space, and use the storage location encoding of the parent node of the character node as the check encoding of the character node to construct the sensitive word library.

[0082] In other words, after determining the basic encoding of each character node in the sensitive word tree, the storage location encoding of the child node of the character node can be set as the sum of the basic encoding of the character node and the character encoding corresponding to the text character of the child node. Furthermore, the storage location encoding of the parent node of the character node is used as the check encoding of the character node. In this way, each character node and its parent node can be associated through the check encoding.

[0083] Specifically, the method further includes: for the lowest-level character node of each path, using the preset end base code as the base code of the lowest-level character node, and using the storage location code of the parent node of the lowest-level character node as the check code of the lowest-level character node, so as to add the lowest-level character node to the sensitive word library.

[0084] In other words, by using a preset end-of-path base code to mark the character node that is the last text character in a path reaching the sensitive word tree, the system informs the user that the last text character of a sensitive word has been found. The lowest-level character node in a path may also be the parent node of other character nodes. In other words, the last text character of a sensitive word may be another text character that is the next text character in the path. For example, in the sensitive word AB, text character B is the last text character, but in the sensitive words ABB and ABC, text character B is not the last text character. Therefore, using the preset end-of-path base code directly as the base code of the lowest-level character node means that finding the sensitive word AB is sufficient to indicate that a sensitive word has been found, without needing to determine whether it is actually AB, ABB, or ABC. Alternatively, if the sensitive word tree does not contain a node that is both the lowest-level character node in a path and the parent node of other character nodes, the preset end-of-path base code can be directly used as the base code of the lowest-level character node.

[0085] Specifically, the sensitive word tree further includes an ending leaf node for indicating the end of a path. The method further includes: for the lowest-level character node of each path, if the lowest-level character node is also the parent node of character nodes of other paths, then setting the ending leaf node for the lowest-level character node, using a preset ending base code as the base code of the ending leaf node, and using the storage location code of the lowest-level character node as the check code of the ending leaf node, so as to add the ending leaf node to the sensitive word library; if the lowest-level character node is not the parent node of character nodes of other paths, then using the preset ending base code as the base code of the lowest-level character node, and using the storage location code of the parent node of the lowest-level character node as the check code of the lowest-level character node, so as to add the lowest-level character node to the sensitive word library.

[0086] In other words, when the lowest-level character node of one path is also the parent node of character nodes in other paths, it is necessary to distinguish the lowest-level character node in this case to differentiate between sensitive words AB and ABB during subsequent queries. Furthermore, when the lowest-level character node is also the parent node of character nodes in other paths, an ending leaf node is set for the lowest-level character node, making the ending leaf node a child node of the lowest-level character node. The basic code of this lowest-level character node is calculated using the aforementioned virtual encoding method. The preset ending basic code is used as the basic code of the ending leaf node, and the storage location code of the lowest-level character node is used as the check code of the ending leaf node. When the lowest-level character node is not the parent node of character nodes in other paths, the ending leaf node of the lowest-level character node does not need to be stored; the preset ending basic code is directly used as the basic code of the lowest-level character node, and the storage location code of the parent node of the lowest-level character node is used as the check code of the lowest-level character node.

[0087] like Figure 2As shown, the first-level character node A, paired with its child node B, directly forms a path in the sensitive word tree to indicate the sensitive word AB. In this case, the child node B of the first-level character node A is the bottom-level character node of this path. The first-level character node A, paired with its child node B and grandchild node B, forms a path in the sensitive word tree to indicate the sensitive word ABB. In this case, the grandchild node B of the first-level character node A is the bottom-level character node of this path. Furthermore, the child node B of the first-level character node A is both the bottom-level character node of one path in the sensitive word tree and the parent node of the bottom-level character node of another path in the sensitive word tree. If only the character node is stored in the sensitive word database, the sensitive word AB will be missing from the database. Therefore, it is necessary to store the ending leaf node 1 of the path corresponding to the sensitive word AB in the sensitive word database. Therefore, when the child nodes of the lowest-level character node include both ending leaf nodes and other character nodes, the ending leaf node among the child nodes of the lowest-level character node is taken as the target ending leaf node. The preset ending base code - 1 can be directly used as the base code of the target ending leaf node, and the storage location code of the lowest-level character node can be used as the check code of the target ending leaf node to add it to the sensitive word database. In this case, it is not necessary to store all ending leaf nodes in the sensitive word database; only the target ending leaf node needs to be added. The lowest-level character nodes of the parent nodes of non-other character nodes in the sensitive word tree include: the lowest-level character node D of the path corresponding to sensitive word AD, the lowest-level character node B of the path corresponding to sensitive word ABB, the lowest-level character node C of the path corresponding to sensitive word ABC, the lowest-level character node B of the path corresponding to sensitive word BCB, the lowest-level character node D of the path corresponding to sensitive word BCD, the lowest-level character node B of the path corresponding to sensitive word CBB, the lowest-level character node D of the path corresponding to sensitive word CBD, and the lowest-level character node C of the path corresponding to sensitive word DAC. Therefore, the preset end base code -1 can be directly used as the base code of the lowest-level character node whose child node is only an end leaf node, and the storage location code of the parent node of the lowest-level character node can be used as the check code of the aforementioned lowest-level character node, so as to add the lowest-level character node whose child node is only an end leaf node to the sensitive word library. Table 5 is obtained in the above manner. Table 5 refers to a double array table obtained by adding each character node and the target end leaf node to the sensitive word library.

[0088] Table 5:

[0089]

[0090] Furthermore, the base encoding of the lowest-level character node whose child nodes are only the ending leaf nodes and the target ending leaf node is set to the preset ending base encoding -1. Thus, the ending of sensitive words is distinguished by the preset ending base encoding -1, so that it is not necessary to add all ending leaf nodes to the sensitive word database, thus saving storage space.

[0091] Specifically, the method further includes: determining the storage location code of the end character of the end leaf node in the storage space based on the end character code corresponding to the end character of the end leaf node and the basic code of its parent node; using the preset end basic code as the basic code of the end leaf node; and using the storage location code of the parent node of the end leaf node as the check code of the end leaf node, so as to add the end leaf node to the sensitive word database.

[0092] In other words, all ending leaf nodes are added to the sensitive word database. For each ending leaf node, the sum of the base code of its parent node and the code of the ending character of the ending leaf node is used as the storage location code of the ending leaf node. A preset ending base code is used as the base code of the ending leaf node, and the storage location code of its parent node is used as the check code of the ending leaf node. This process adds each ending leaf node to the sensitive word database, and the preset ending base code is used to distinguish between ending leaf nodes. For example, please refer to Table 6, which is a double-array table obtained by adding each character node and all ending leaf nodes to the sensitive word database.

[0093] Table 6:

[0094]

[0095] Furthermore, each character node and the final leaf node of the sensitive word tree can be stored layer by layer into the sensitive word database. The parent-child relationship of each node can be associated and recorded through basic encoding and check encoding. In the same way, subsequent nodes can be inserted into the empty positions of the table, thereby improving storage efficiency and reducing storage volume.

[0096] Specifically, the sensitive word database also includes combined sensitive words, and the method further includes: setting the character node corresponding to the target text character of the combined word in each combined sensitive word as the first matching number, and setting the character node corresponding to the text character other than the target text character of the combined word as the second matching number, wherein the first matching number is greater than the second matching number, and the matching number is used to indicate the number of times the query character matches the text character of the corresponding character node stored in the sensitive word database.

[0097] The combination of sensitive words includes at least one first text character, a random number of random characters, and at least one second text character arranged sequentially. The first and second text characters are text characters extracted from the sensitive word. The random characters refer to any characters, which can be text characters from a dictionary or characters from a dictionary, and the random number is an integer greater than 0. The target text character refers to the first text character closest to the random characters, or the last first text character. The first match count refers to the maximum order of the first second text character closest to the random characters relative to the last first text character. In other words, if the first second text character exists within the first match count after the last first text character, it is considered to potentially contain a combination of sensitive words.

[0098] For example, if the combined sensitive word can be AB….C, then if the text character C appears within 10 characters after identifying text characters A and B, it is considered to be the combined sensitive word AB….C. Furthermore, in the combined sensitive word AB….C, the first text character includes text characters A and B, the target text character is text character B, the first matching count is 10, and the second text character is text character C. That is, after identifying the target text character B, if the text character C is identified within 10 matching counts, it is considered to be the combined sensitive word AB….C.

[0099] For example, it is only necessary to add a combination code to each node in the double-array table. The combination code reflects the number of matches for that node as either the first match count or the second match count. The combination code of the target text character of the combined sensitive word is set to the first match count, and the combination code of the text characters other than the target text character is set to the second match count, where the second match count can be set to 1. Furthermore, by adding combination codes to Table 5, Table 7 is obtained. Table 7 refers to another double-array table obtained by adding each character node and the target end leaf node to the sensitive word database.

[0100] Table 7:

[0101]

[0102] Based on the same application concept, this application also provides a word query method corresponding to the lexicon construction method provided in the above embodiments. Since the word query method in this application solves the problem in a similar way to the lexicon construction method in the above embodiments of this application, the implementation of the word query method can refer to the implementation of the lexicon construction method, and the repeated parts will not be described again.

[0103] The word query method based on a dual-array tree provided in this application uses the sensitive word library created by the method described in the previous embodiments. Specifically, a word query method based on a dual-array tree includes: determining whether the first query character in the sensitive word to be queried is a dictionary-encoded text character; if so, determining the first target storage location code of the first query character in the storage space based on the preset basic encoding of the root node of the sensitive word library and the character encoding corresponding to the first query character of the sensitive word to be queried; determining whether the text character stored at the storage location is the first query character, and the corresponding detection character, based on the first target storage location code. Check if the encoding is the storage location encoding of the root node; if so, determine if the base encoding corresponding to the first target storage location encoding is the preset end base encoding; if not, based on the base encoding corresponding to the first target storage location encoding and the character encoding corresponding to the second query character of the sensitive word to be searched, determine the second target storage location encoding of the second query character in the storage space; according to the text character stored in the storage location corresponding to the current target storage location encoding, the base encoding and the check encoding, determine whether the current query character meets the sensitive word character determination conditions; if all conditions are met, then the sensitive word to be searched is determined to be a sensitive word stored in the sensitive word library.

[0104] The sensitive words to be searched can be extracted sequentially from a sentence using a moving window, the size of which can vary. Alternatively, the sensitive words to be searched can be obtained by randomly selecting adjacent characters from a sentence, or by identifying at least one sensitive word using a pre-defined word segmentation dictionary. In this way, multiple sensitive words to be searched can be extracted from a sentence, and these multiple sensitive words can be processed in parallel to identify whether each sensitive word is a sensitive word.

[0105] Here, the first query character refers to the first query character in the sensitive word to be searched. That is to say, it must be determined at least whether the first query character of the sensitive word to be searched is a text character that enters the dictionary encoding. If the first query character is a text character that enters the dictionary encoding, it is also necessary to determine whether the first query character is the text character indicated by the first-level character node of the sensitive word tree (or the text character indicated by the child node of the root node of the sensitive word tree). When it is determined that the first query character is the text character indicated by the first-level character node of the sensitive word tree, the first target storage location encoding of the first query character can be calculated according to the sum of the basic encoding preset by the root node of the sensitive word library and the character encoding corresponding to the first query character. Then, a double array table is used to query whether the text character stored at the storage location corresponding to the first target storage location encoding is the first query character, and to query whether the check encoding corresponding to the first target storage location is the storage location encoding of the root node. This double confirmation confirms whether the first query character is a first-level character node of the sensitive word tree.Therefore, when it is determined that the text character stored at the storage location corresponding to the first target storage location code is the first query character, and the check code corresponding to the first target storage location is the storage location code where the root node is located, it is then determined whether the basic code corresponding to the first target storage location code is the preset end basic code -1, or whether the storage location code indicated by the sum of the basic code corresponding to the first target storage location code and the end character code 0 corresponds to the end character code 0; when the basic code corresponding to the first target storage location code is the preset end basic code -1, or when it is determined that the storage location code indicated by the sum of the basic code corresponding to the first target storage location code and the end character code 0 corresponds to the end character code 0, it is considered that the node indicated by the first target storage location code is the last level character node of the sensitive word, which means that the first query character is a sensitive word; if the basic code corresponding to the first target storage location code is not the preset end basic code -1, and the storage location code indicated by the sum of the basic code corresponding to the first target storage location code and the end character code 0 corresponds to the end character code 0, it is considered that the node indicated by the first target storage location code is the last level character node of the sensitive word, which means that the first query character is a sensitive word; if the basic code corresponding to the first target storage location code is not the preset end basic code -1, and the storage location code indicated by the sum of the basic code corresponding to the first target storage location code and the end character code 0 corresponds to the end character code 0, it is considered that the node indicated by the first target storage location code is the last level character node of the sensitive word, which means that the first query character is a sensitive word; If the storage location does not correspond to the ending character code 0, it is considered that the next query character (the second query character) of the first query character needs to be queried in sequence. Therefore, by calculating the sum of the basic code corresponding to the first target storage location code and the character code corresponding to the second query character, the second target storage location code of the second query character in the storage space is obtained. Then, the text characters stored in the double array table are queried again through the second target storage location code to see if they are consistent with the second query character, and it is determined whether the check code corresponding to the second target storage location code is the first target storage location code. Thus, when the text characters stored in the double array table are consistent with the second query character and the check code corresponding to the second target storage location code is the first target storage location code, it is considered that the child node of the first query character in the sensitive word tree includes the second query character. And so on, each query character of the sensitive word to be queried is matched in turn, and it is determined that each query character is matched and the query character of the sensitive word to be queried includes the lowest level text character of a path in the sensitive word tree. Then, the sensitive word to be queried is considered to be a sensitive word stored in the sensitive word library.

[0106] For example, if the sensitive word to be searched is ABC, firstly, the character encoding of the first query character A in the dictionary is determined to be 1. The sum of character encoding 1 of A and the preset base encoding 1 of the root node is calculated to be 2. The first target storage location encoding of the query character A is considered to be 2. The character stored at the storage location corresponding to storage location encoding 2 in Table 5 is found to be A, which matches the query character A. The check encoding corresponding to storage location encoding 2 is 0. Check encoding 0 is the storage location encoding of the root node. Thus, it is determined that the query character A matches the first-level character node A of the sensitive word tree. Next, the character encoding of the second query character B is determined to be 2. The sum of character encoding 2 of the second query character B and the base encoding 4 of the first-level character node A is calculated to be 6. The second target storage location encoding of the second query character B is considered to be 6. The character stored at the storage location corresponding to storage location encoding 6 in Table 5 is found to be B, which matches the second query character B. The check encoding corresponding to storage location encoding 6 is 2. Check encoding 2 matches the first target storage location encoding 2. Thus, it is determined that the second query character B... The first query character B is a child node of the first-level character node A in the sensitive word tree, meaning the second query character B is a second-level character node B in the sensitive word tree. The character code of the third query character C is determined to be 3, and the sum of the character code 3 of the third query character C and the basic code 13 of the second-level character node B is calculated to be 16. Therefore, the third target storage location code of the third query character C is considered to be 16. From Table 5, the character stored at the storage location corresponding to storage location code 16 is C, which matches the third query character C. The check code corresponding to storage location code 16 is 6, which matches the second target storage location code 6. Thus, it is determined that the third query character C is a grandchild node of the first-level character node A in the sensitive word tree and a child node of the second-level character node B in the sensitive word tree, meaning the third query character C is a third-level character node C in the sensitive word tree. Furthermore, the basic code corresponding to the third target storage location code 16 is the preset end basic code -1. Therefore, the third-level character node C is considered to be the lowest-level character node of the sensitive word, thus confirming that the sensitive word ABC exists in the sensitive word database.

[0107] For example, if the sensitive word to be searched is BCA, firstly, the character encoding of the first query character B in the dictionary is determined to be 2. The sum of the character encoding 2 of B and the preset basic encoding 1 of the root node is calculated to be 3. The first target storage location encoding of the query character B is considered to be 3. The character stored at the storage location corresponding to storage location encoding 3 is found in Table 5 to be B, which matches the query character B. The check encoding corresponding to storage location encoding 3 is 0. Check encoding 0 is the storage location encoding of the root node. Thus, it is determined that the query character B matches the first-level character node B of the sensitive word tree. Next, the character encoding of the second query character C is determined to be 3. The sum of the character encoding 3 of the second query character C and the basic encoding 4 of the first-level character node B is calculated to be 7. The second target storage location of the second query character C is considered to be... The code is set to 7. The character stored at the storage location corresponding to code 7 in Table 5 is C, which matches the second query character C. The check code corresponding to storage location code 7 is 3, which matches the first target storage location code 3. Therefore, it is determined that the second query character C is a child node of the first-level character node B in the sensitive word tree, i.e., the second query character C is the second-level character node C in the sensitive word tree. The character code of the third query character A is again determined to be 1, and the sum of the character code 1 of the third query character A and the basic code 16 of the second-level character node C is calculated to be 17. It is considered that the third target storage location code of the third query character A is 17. The storage location corresponding to storage location code 17 in Table 5 is empty. Therefore, it is determined that the sensitive word BCA to be searched is not in the sensitive word database.

[0108] Specifically, the method further includes: if the first part of the query characters of the sensitive word to be searched meets the sensitive word character determination condition, then based on the character encoding of the second part of the query characters of the sensitive word to be searched and the basic encoding corresponding to the target storage location encoding of the last query character in the first part of the query characters, the target query location encoding in the sensitive word library is determined; based on the second part of the query characters and the text characters stored in the storage location corresponding to the target query location encoding, it is sequentially determined whether they match, and the number of matches is recorded; if the number of matches is less than or equal to a preset first number of matches, and the third part of the query characters in the second part of the query characters meets the sensitive word character determination condition, then the sensitive word to be searched is determined as a combined sensitive word stored in the sensitive word library.

[0109] The sensitive word character determination condition refers to the fact that each query character matches the double array table. In other words, each query character can determine its target query position code according to its corresponding character code and the basic code of the previous query character. Furthermore, the character stored at the corresponding storage position of the target query position code in the double array table is consistent with the query character, and the check code corresponding to the target query position code is the query position code of the previous query character.

[0110] The first part of the query characters refers to the n query characters in the first order of the sensitive word to be searched that match the double array table; the second part of the query characters refers to the m query characters after the first part of the query characters that do not match the double array table; and the third part of the query characters refers to the p query characters in the second order of the sensitive word to be searched that match the double array table.

[0111] In other words, when identifying sensitive words to be searched, the combined encoding of each text character of the sensitive word to be searched is determined. When the combined encoding is 1, the text character only needs to be matched once. If the match is successful, the next text character is matched. If the match is unsuccessful, the sensitive word to be searched is considered not to be a sensitive word in the sensitive word tree. When the combined encoding is 10, 10 matches can be performed. If the match is unsuccessful once, the next text character can be searched. If the match is successful within 10 matches, the matching continues. If the match is unsuccessful in 10 matches, the sensitive word to be searched is considered not to be a sensitive word in the sensitive word tree.

[0112] For example, if the sensitive word to be searched is BCA, firstly, the character encoding of the first query character B in the dictionary is determined to be 2. The sum of the character encoding 2 of B and the preset base encoding 1 of the root node is calculated to be 3. The first target storage location encoding of the query character B is considered to be 3. The character stored at the storage location corresponding to storage location encoding 3 is found in Table 7 and is B, which matches the query character B. The check encoding corresponding to storage location encoding 3 is 0. Check encoding 0 is the storage location encoding of the root node. Thus, it is determined that the query character B matches the first-level character node B of the sensitive word tree. Next, the character encoding of the second query character C is determined to be 3. The sum of the character encoding 3 of the second query character C and the base encoding 4 of the first-level character node B is calculated to be 7. The second target storage location encoding of the second query character C is considered to be 7. The storage location encoding 7 is found in Table 7. The character stored at the corresponding storage location is C, which is consistent with the second query character C. The check code corresponding to storage location code 7 is 3, which is consistent with the first target storage location code 3. Therefore, it is determined that the second query character C is a child node of the first-level character node B in the sensitive word tree, that is, the second query character C is the second-level character node C in the sensitive word tree. It is further determined that the character code of the third query character A is 1, and the sum of the character code 1 of the third query character A and the basic code 16 of the second-level character node C is 17. It is considered that the third target storage location code of the third query character A is 17. The storage location corresponding to storage location code 17 in Table 7 is empty, and the combined code of the second target storage location code 7 is 1, which is not a combined sensitive word. Therefore, it can only be queried once. Therefore, it is determined that the sensitive word BCA to be queried is not in the sensitive word database.

[0113] For example, if the sensitive word to be searched is ABDAC, firstly, the character encoding of the first query character A in the dictionary is determined to be 1, and the sum of the character encoding 1 of A and the preset basic encoding 1 of the root node is calculated to be 2. The first target storage location encoding of the query character A is considered to be 2. The character stored at the storage location corresponding to storage location encoding 2 is found in Table 7 to be A, which matches the query character A. The check encoding corresponding to storage location encoding 2 is 0. Check encoding 0 is the storage location encoding of the root node. Thus, it is determined that the query character A matches the first-level character node A of the sensitive word tree. Next, the character encoding of the second query character B is determined to be 2, and the sum of the character encoding of the second query character B is calculated to be 2. The sum of character encoding 2 and the base encoding 4 of the first-level character node A is 6. Therefore, the second target storage location encoding of the second query character B is considered to be 6. Looking up the storage location encoding 6 in Table 7, the character stored at that location is B, which matches the second query character B. The check encoding corresponding to storage location encoding 6 is 2, which matches the first target storage location encoding 2. Therefore, it is determined that the second query character B is a child node of the first-level character node A in the sensitive word tree, i.e., the second query character B is the second-level character node B in the sensitive word tree. The character encoding of the third query character D is determined to be 4, and the sum of character encoding 4 of the third query character D and the base encoding 4 of the second-level character node A is calculated. The sum of the base code 13 of point B is 17. It is assumed that the third target storage location code of the third query character D is 17. However, the storage location corresponding to code 17 in Table 7 is empty, indicating a mismatch in the double array table. Therefore, the combined code of the second-level character node B is 10, allowing for a second match. Thus, the second query character B is the last query character in the first part of the query characters. The sum of the character code 1 of the fourth query character A and the base code 13 of the second-level character node B is 14. The storage location corresponding to code 14 of the fourth target in Table 7 is empty, indicating a mismatch in the double array table. Furthermore, the current matching count of 2 is less than the second-level character... If the combined code of node B is 10, then continue with the third matching. The sum of the character code 3 of the fifth query character C and the basic code 13 of the second-level character node B is determined to be 16. From Table 7, the character stored at the storage location corresponding to storage location code 16 is C, which matches the fifth query character C. The check code corresponding to storage location code 16 is 6, which matches the second target storage location code 6. Furthermore, the basic code corresponding to storage location code 16 is -1. Therefore, the fifth query character C is determined to be the lowest-level character node of the sensitive word, satisfying the end-matching condition. Thus, the sensitive word ABDAC is determined to be the combined sensitive word AB......C.

[0114] Based on the same application concept, this application also provides a double-array tree lexicon construction device corresponding to the double-array tree lexicon construction method provided in the above embodiments. Since the principle of the device in this application is similar to the double-array tree lexicon construction method in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0115] like Figure 3 As shown, Figure 3 This application provides a functional block diagram of a dictionary construction device based on a double-array tree. The double-array tree-based dictionary construction device 10 includes: a sensitive word tree generation module 101, which generates a sensitive word tree based on the text characters split from each sensitive word and the arrangement order of these text characters, where each path in the sensitive word tree corresponds to a sensitive word; an encoding module 102, which performs dictionary encoding on the text characters split from the sensitive word after deduplication, obtaining character codes corresponding to each text character; a basic encoding determination module 103, which sequentially determines the basic encoding of each character node in the sensitive word tree based on at least one of random selection trial calculation, auto-incrementing selection trial calculation, overall query selection trial calculation, and preset data selection trial calculation; and a construction module 104, which determines the storage location encoding of the text character of each character node in the storage space based on the character code corresponding to the text character of each character node and the basic encoding of its parent node, and uses the storage location encoding of the parent node of the character node as the check encoding of the character node, thereby constructing the sensitive word library.

[0116] Based on the same application concept, this application also provides a word query device corresponding to the double-array tree lexicon construction method provided in the above embodiments. Since the principle of the device in this application is similar to the double-array tree lexicon construction method in the above embodiments of this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0117] This application embodiment also provides a word query device based on a dual-array tree. This device uses the sensitive word database created in the above embodiments. The device includes: a first determining module, which determines whether a first query character in the sensitive word to be queried is a dictionary-encoded text character; a second determining module, which, if so, determines a first target storage location code for the first query character in the storage space based on the preset basic encoding of the root node of the sensitive word database and the character encoding corresponding to the first query character of the sensitive word to be queried; and a checking module, which determines whether the text character stored at the first target storage location is the first query character, and whether the checking code corresponding to the first target storage location code is the root node, based on the first target storage location code. The storage location code is determined by the following modules: A first target storage location code is assigned a base code; a second target storage location code is assigned a base code; a third determination module is assigned a base code, which determines the second target storage location code of the second query character in the storage space based on the base code corresponding to the first target storage location code and the character code corresponding to the second query character of the sensitive word; a third determination module is assigned a condition comparison module, which determines whether the current query character meets the sensitive word character determination conditions based on the text character stored in the storage location corresponding to the current target storage location code, the base code, and the check code; and a fourth determination module is assigned a sensitive word, which determines the sensitive word to be searched as a sensitive word stored in the sensitive word database if all conditions are met.

[0118] Furthermore, this application's dual-array tree storage method improves storage utilization and reduces storage space usage compared to directly storing the sensitive word tree. Moreover, it improves dictionary construction efficiency by querying the storage location of character nodes with a large number of tightly arranged child nodes as a whole. During queries, the storage location code and check code of each query character are used to double-confirm whether it is stored in the dual-array tree, and the methods for calculating the storage location code and check code are relatively simple. This greatly improves the efficiency of sensitive word queries; additionally, combined encoding is set up to query for combined sensitive words, expanding the scope of application.

[0119] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0121] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0122] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing a dictionary based on a double array tree, characterized by, The method comprises: generating a sensitive word tree based on the split text characters of each sensitive word and the arrangement order of each text character, each path in the sensitive word tree corresponding to a sensitive word; performing dictionary coding on the split text characters of the sensitive word after deduplication to obtain character codes corresponding to all text characters respectively; determining the base code of each character node in the sensitive word tree in layers according to at least one of random selection trial, self-increment selection trial, overall query selection trial and preset data selection trial, wherein the random selection trial is to randomly set the base code for each character node, the self-increment selection trial is to set the base code for each character node in ascending order of integer, the overall query selection trial is to store and set the base code for a plurality of character nodes as a whole, and the preset data selection trial is to select the base code for each character node from each preset number in a preset data set; determining the storage location code of the text character of the character node in the storage space based on the character code corresponding to the text character of each character node and the base code of the parent node of the character node, and taking the storage location code of the parent node of the character node as the check code of the character node, to construct a sensitive word library; wherein the base code of each character node in the sensitive word tree is determined in layers according to a preset trial rule, and the storage space corresponding to at least one storage location code required by all child nodes of the character node is determined through the virtual code of the character node and the character code of all child nodes of the character node; when each storage location corresponding to each storage location code is idle, the virtual code corresponding to each storage location is taken as the base code of the character node. The virtual code of the character node is set according to a preset trial rule, which comprises:

2. The method of claim 1, wherein, determining whether the overall query condition is met according to the number of child nodes of the character node and the character code corresponding to each child node; when the overall query condition is met, the value of the virtual code is determined according to the difference between the minimum character code and the maximum character code of the child nodes that meet the overall query condition. When the overall query condition is met, the value of the virtual code is determined according to the difference between the minimum character code and the maximum character code of the child nodes that meet the overall query condition, which comprises:

3. The method of claim 2, wherein, if all child nodes of the character node meet the overall query condition, the storage location code corresponding to the minimum character code of the child nodes is taken as the starting point of the storage location code corresponding to the storage space required by all child nodes of the character node; the storage space from the storage location code corresponding to the minimum character code of all child nodes to the storage location code corresponding to the maximum character code is taken as a whole to determine whether it is idle; if yes, the current virtual code corresponding to each storage location when each storage location corresponding to each storage location code is idle is taken as the base code of the character node; if no, the value of the next round of virtual code is set to at least increase the difference between the maximum character code and the minimum character code of all child nodes plus 1 on the current virtual code. ​ 4. The method of claim 2, wherein, The value of the virtual code is determined according to a difference between the minimum character code and the maximum character code in the child nodes satisfying the overall query condition when the overall query condition is satisfied, and the value of the virtual code comprises: If part of the child nodes of the character node satisfy the overall query condition, the minimum character code in the child nodes satisfying the overall query condition is taken as the starting point of the storage location code corresponding to the required storage space of the part of the child nodes of the character node; The storage space from the storage location code corresponding to the minimum character code to the storage location code corresponding to the maximum character code in the child nodes satisfying the overall query condition is taken as the storage space for judging whether the overall storage space is idle, and the storage location code corresponding to the required storage space of the other child nodes of the character node is determined based on the current virtual code and the character code of the child node of the character node not satisfying the overall query condition, and whether the storage location corresponding to each storage location code is idle is judged; If yes, the current virtual code corresponding to the storage location in which all the storage locations corresponding to each storage location code are idle is taken as the basic code of the character node; if no, the value of the next round of virtual code is set to be at least the difference between the maximum character code and the minimum character code in the child nodes satisfying the overall query condition plus 1 on the current virtual code.

5. The method of claim 2, wherein, The method further comprises: When the overall query condition is not satisfied, the storage location code corresponding to the required storage space of each child node of the character node is determined based on the current virtual code and the character code of each child node of the character node, and whether the storage location corresponding to each storage location code is idle is judged; If yes, the current virtual code corresponding to the storage location in which all the storage locations corresponding to each storage location code are idle is taken as the basic code of the character node; if no, the value of the next round of virtual code is set to be a preset value on the current virtual code, or the value of the next round of virtual code is randomly and repeatedly obtained within a preset value range, or the value of the next round of virtual code is obtained according to a preset value sequence.

6. The method of claim 1, wherein, The method further comprises: For the bottom character node of each path, a preset ending basic code is taken as the basic code of the bottom character node, and the storage location code of the parent node of the bottom character node is taken as the check code of the bottom character node, so that the bottom character node is added to the sensitive word library.

7. The method of claim 1, wherein, The sensitive word tree further comprises an ending leaf node for indicating the end of a path, and the method further comprises: For the bottom character node of each path, if the bottom character node is also the parent node of the character node of another path, the ending leaf node is set for the bottom character node, a preset ending basic code is taken as the basic code of the ending leaf node, and the storage location code of the bottom character node is taken as the check code of the ending leaf node, so that the ending leaf node is added to the sensitive word library. If the bottommost character node is not a parent node of any other path character node, a preset end base code is set as a base code of the bottommost character node, a storage location code of a parent node of the bottommost character node is set as a check code of the bottommost character node, and the bottommost character node is added to the sensitive word library.

8. The method of claim 1, wherein, The sensitive word tree further comprises end leaf nodes for indicating an end of a path, each end leaf node comprising an end character, and the method further comprises: performing dictionary encoding on the end character to obtain an end character code corresponding to the end character; determining a storage location code of the end character of the end leaf node in the storage space according to the end character code corresponding to the end character of the end leaf node and a base code of a parent node of the end leaf node, setting a preset end base code as a base code of the end leaf node, and setting the storage location code of the parent node of the end leaf node as a check code of the end leaf node, so as to add the end leaf node to the sensitive word library.

9. The method of claim 1, wherein, The sensitive word library further comprises combined sensitive words, and the method further comprises: setting a first matching number for a character node corresponding to a target text character of a combined word in each combined sensitive word and setting a second matching number for a character node corresponding to a text character other than the target text character of the combined word, wherein the first matching number is greater than the second matching number, and the matching number is used to indicate a matching number of a query character and a text character of a corresponding character node stored in the sensitive word library.

10. A method of word lookup based on a dual array tree, characterized by The method applies the sensitive word library created by the method of any one of claims 1 to 9, and the method comprises: determining whether a first query character in a sensitive word to be checked is a dictionary-encoded text character; if yes, determining a first target storage location code of the first query character in the storage space based on a preset base code of a root node of the sensitive word library and a character code corresponding to the first query character of the sensitive word to be checked; determining whether a text character stored in the storage location corresponding to the first target storage location code is the first query character and whether a check code corresponding to the first target storage location code is a storage location code of the root node according to the first target storage location code; if yes, determining whether a base code corresponding to the first target storage location code is a preset end base code; if no, determining a second target storage location code of a second query character in the storage space based on the base code corresponding to the first target storage location code and a character code corresponding to the second query character of the sensitive word to be checked; determining whether the current query character meets a sensitive word character determination condition according to the text character stored in the storage location corresponding to the current target storage location code, the base code, and the check code; if yes, determining that the sensitive word to be checked is a sensitive word stored in the sensitive word library.

11. The method of claim 10, wherein, The method further comprises: If the first part of the query character of the sensitive word to be checked meets the sensitive word character determination condition, the target query position code in the sensitive word library is determined based on the character encoding of the second part of the query character of the sensitive word to be checked and the basic code corresponding to the target storage position code of the last query character in the first part of the query character. Based on the text characters stored in the storage position corresponding to the target query position code and the second part of the query character, it is determined in turn whether they match, and the number of matches is recorded. If the number of matches is less than or equal to a preset first number of matches, and the third part of the query character in the second part of the query character meets the sensitive word character determination condition, the sensitive word to be checked is determined as a combined sensitive word stored in the sensitive word library.

12. A two-array tree-based dictionary construction apparatus, characterized by comprising: The device comprises: A sensitive word tree generation module generates a sensitive word tree based on the text characters split by each sensitive word and the arrangement order of each text character. Each path in the sensitive word tree corresponds to a sensitive word. An encoding module performs dictionary encoding on the text characters split by the sensitive word after removing duplicates to obtain the character encoding corresponding to each text character. A basic code determination module determines the basic code of each character node in the sensitive word tree in layers in turn according to at least one of random selection trial, self-increment selection trial, overall query selection trial, and preset data selection trial. The random selection trial randomly sets the basic code for each character node. The self-increment selection trial sets the basic code for each character node in ascending order. The overall query selection trial stores and sets the basic code for a plurality of character nodes as a whole. The preset data selection trial selects the basic code for each character node from each preset number in a preset data set. A construction module determines the storage position code of the text character of each character node in the storage space based on the character encoding corresponding to the text character of each character node and the basic code of the parent node of each character node, and takes the storage position code of the parent node of each character node as the check code of each character node, to construct a sensitive word library. The basic code determination module determines the basic code of each character node in the sensitive word tree in layers in turn by: setting a virtual code for each character node according to a preset trial rule, and determining at least one storage position code corresponding to the storage space required by all child nodes of each character node through the virtual code and the character encoding of all child nodes of each character node; and taking the virtual code corresponding to each storage position code as the basic code of each character node when each storage position corresponding to each storage position code is empty.

13. A word query device based on a dual array tree, characterized by The device applies the sensitive word library created by the method of any one of claims 1 to 9, and the device comprises: A first determination module determines whether the first query character in the sensitive word to be checked is a dictionary-encoded text character. A second determination module determines the first target storage position code of the first query character in the storage space based on the preset basic code of the root node of the sensitive word library and the character encoding corresponding to the first query character of the sensitive word to be checked if the first query character is a dictionary-encoded text character. The checking module determines whether the text character stored in the storage location corresponding to the first target storage location code is the first query character and whether the checking code corresponding to the first target storage location code is the storage location code of the root node according to the first target storage location code; The judging module judges whether the base code corresponding to the first target storage location code is the preset ending base code if yes; The third determining module determines the second target storage location code of the second query character in the storage space based on the base code corresponding to the first target storage location code and the character code corresponding to the second query character of the sensitive word to be checked if no; The condition comparison module determines whether the current query character meets the sensitive word character determination condition according to the text character stored in the storage location corresponding to the current target storage location code, the base code and the checking code; The fourth determining module determines the sensitive word to be checked as the sensitive word stored in the sensitive word library if all meet.

Images