Text content verification method, device, electronic device and storage medium
By constructing a black and white list state transfer tree, each text in the text is sequentially checked, the problems of inaccurate word segmentation and waste of resources in the prior art are solved, and efficient and convenient sensitive vocabulary detection is achieved.
Patent Information
- Application Number
- CN202110126051.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-29
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2041-01-29
AI Technical Summary
The prior art has problems of word segmentation inaccuracy and waste of resources when detecting sensitive words in text, resulting in inefficient verification.
By constructing a blacklist state transfer tree and a whitelist state transfer tree, input each text of the target text into these trees in turn, determine the node hit result, and determine the processing method of the next text based on the node hit result, and finally determine whether the text contains black and white list words.
It improves the accuracy and efficiency of text verification, reduces resource waste, and achieves efficient and convenient sensitive vocabulary detection.
Smart Images

Figure CN114091435B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a text content verification method, device, electronic device, and storage medium. Background Art
[0002] Currently, text management is an important means of internet risk management. Text risk management primarily involves detecting whether a text contains sensitive words, thereby determining the text risk based on these words.
[0003] Currently, the method for detecting the presence of sensitive words in a text is to first segment each sentence to obtain at least one matching word; then, each matching word is matched against a blacklist or whitelist word, and finally, the matching results are used to determine whether it is a blacklist or whitelist word. To further improve the accuracy of identifying sensitive words, special settings can be made to the word segmenter, such as setting a higher weight for blacklist words. This will ensure that blacklist words are grouped together as much as possible during the word segmentation process, thereby ensuring that blacklist words are found in the sentence.
[0004] Another way to monitor whether there are sensitive words in the text is: divide each sentence in the text into at least one word to be matched, and then match the whitelist words and blacklist words respectively. That is, each word to be matched in the sentence needs to be traversed through the blacklist words and whitelist words in turn, that is, identified twice.
[0005] When implementing the technical solution based on the above method, the inventors found the following problems:
[0006] After setting the weight value of the blacklist words in the word segmenter, there is a technical problem that the words used to form the sentence are inaccurate, resulting in inaccurate blacklist words; further, when the words to be matched are input into the blacklist and whitelist for verification respectively, there are technical problems of low efficiency and waste of resources. Summary of the Invention
[0007] The present invention provides a text content verification method, device, electronic device and storage medium, so as to achieve the technical effect of improving text verification efficiency and resource utilization while accurately verifying text content.
[0008] In a first aspect, an embodiment of the present invention provides a text content verification method, the method comprising:
[0009] Obtain the first character of the target text and use the first character as the current text;
[0010] Determine a root node hit result corresponding to the current text based on a pre-built blacklist state transition tree and a pre-built whitelist state transition tree;
[0011] Determine a target processing method for the next character of the current character based on the root node hit result, and determine a hit result for the next character based on the target processing method;
[0012] The verification result of the target text is determined according to the hit result of each character in the target text.
[0013] In a second aspect, an embodiment of the present invention further provides a text content verification device, the device comprising:
[0014] A text extraction module is used to obtain the first character of the target text and use the first character as the current text;
[0015] A root node matching module, configured to determine a root node hit result corresponding to the current text based on a pre-built blacklist state transition tree and a pre-built whitelist state transition tree;
[0016] a target processing mode determination module, configured to determine a target processing mode for a next character of the current character based on the root node hit result, and determine a hit result for the next character based on the target processing mode;
[0017] The verification result determination module is used to determine the verification result of the target text according to the hit result of each character in the target text.
[0018] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising:
[0019] one or more processors;
[0020] a storage device for storing one or more programs,
[0021] When the one or more programs are executed by the one or more processors, the one or more processors implement the text content verification method as described in any one of the embodiments of the present invention.
[0022] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute the text content verification method as described in any one of the embodiments of the present invention.
[0023] The technical solution of the embodiment of the present invention is to input each character of the target text into a pre-trained black and white list state transition tree in sequence after obtaining the target text, obtain the node hit result corresponding to the current character, and determine the target processing method for the next character processing based on the node hit result, and then determine the existence of black and white list words in the target text based on the node hit result of each character, thereby improving the accuracy of target text detection while improving the text verification efficiency and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings introduced here only illustrate some of the embodiments to be described by the present invention, and are not exhaustive. A person skilled in the art can derive other drawings based on these drawings without inventive effort.
[0025] Figure 1 A flowchart of a text content verification method provided in Example 1 of the present invention;
[0026] Figure 2 A schematic diagram of a state transition tree constructed according to the first embodiment of the present invention;
[0027] Figure 3 A flowchart of a text content verification method provided in the second embodiment of the present invention;
[0028] Figure 4 A schematic diagram of the system structure for executing the text content verification method provided in the third embodiment of the present invention;
[0029] Figure 5 A matching diagram provided in the third embodiment of the present invention;
[0030] Figure 6 A matching diagram provided in the third embodiment of the present invention;
[0031] Figure 7 A schematic diagram of the structure of a text content verification device provided in the fourth embodiment of the present invention;
[0032] Figure 8 This is a structural diagram of an electronic device provided in Example 5 of the present invention. DETAILED DESCRIPTION
[0033] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.
[0034] Example 1
[0035] Figure 1 This is a flowchart of a text content verification method provided in Example 1 of the present invention. This embodiment is applicable to determining whether text content includes whitelisted and blacklisted words. The method can be performed by a text content verification device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, which can be a mobile terminal, a PC, etc. The execution of this technical solution can be performed by a server, a terminal device, or a server and a terminal device in combination.
[0036] Before introducing the technical solution of this embodiment, an example application scenario is first described. In the field of the Internet, text communication is often required. During text communication, the technical solution disclosed in this invention can be used to determine whether the text sent contains sensitive words, thereby determining whether it can be sent to the corresponding terminal device.
[0037] like Figure 1 The method of this embodiment includes:
[0038] S110: Obtain the first character of the target text, and use the first character as the current text.
[0039] Among them, the target text can be a text composed of text sent from the current terminal to other servers or other terminals; or it can be a text to be received from other terminal devices or servers. The text includes at least one character. For example, on a certain platform, a user edits the corresponding content in the content editing control and triggers the "Send" control to send the content edited in the content editing control to other terminals. The text edited in the content editing control can be used as the target text. When sending the target text to other terminals, the server can first obtain the target text and perform text detection on the content of the target text to determine whether the target text includes blacklist words and / or whitelist words. Blacklist words are mainly sensitive words. For example, sensitive words can be transfers, campus loans, etc. Whitelist words are mainly normal words. The first word is the first word in the target text, and the word currently being processed is used as the current word. For example, if the first word is currently being processed, the first word is the current word; if the third to last word of the text is currently being processed, the third to last word is the current word.
[0040] Specifically, after obtaining the target text, in order to determine whether the target text includes blacklist words and / or whitelist words, the first character (the first word) in the target text can be obtained as the current character for verification processing of the first character.
[0041] S120. Based on the pre-constructed blacklist state transition tree and whitelist state transition tree, determine the root node hit result corresponding to the current character.
[0042] Among them, the blacklist state transition tree is a tree diagram established according to the pre-determined blacklist words. Correspondingly, the whitelist state transition tree is a tree diagram established according to the pre-determined whitelist words. The tree diagram includes at least one root node and at least one leaf node associated with each root node. The tree diagram includes at least one branch, and each branch includes at least one leaf node. The number of all leaf nodes represents the depth value of the branch. Since the tree diagram is created based on blacklist words or whitelist words, each node corresponds to a corresponding character. If the root node of the blacklist state transition tree and / or whitelist state transition tree includes the current character, then the corresponding to the current text is root node hit; if the root node of the blacklist state transition tree and / or whitelist state transition tree does not include the current character, then the corresponding to the current text is not hitting the root node. That is, the root node hit result includes hitting the root node and not hitting the root node. For example, if the current character is "day", and at least one root node of the blacklist state transition tree and / or whitelist state transition tree includes "day", then the current character is root node hit.
[0043] Specifically, after obtaining the first character, it can be determined whether the characters corresponding to the root nodes of the blacklist state transition tree and the whitelist state transition tree are the same as the current character. If so, it means hitting the root node; otherwise, it means not hitting the root node.
[0044] It should be noted that in the initial state, the states corresponding to each node in the state transition tree are idle states. Among them, the situation where each time a text is obtained and it is necessary to determine whether the text includes black and white list words can be used as the initial state.
[0045] In this embodiment, the determining the root node hit result corresponding to the current character based on the pre-constructed blacklist state transition tree and whitelist state transition tree includes: determining the target root node that matches the current character according to the target characters corresponding to the root nodes in the blacklist state transition tree and the whitelist state transition tree, and using the target root node as the root node hit result of the current character.
[0046] After obtaining the current text, if the current text is the first character or the text before the current text does not hit the root node, it is possible to continue to determine whether the current text matches the target text corresponding to each root node of the state transition tree. If so, the root node corresponding to the target text identical to the current text is used as the root node hit result of the current text. At the same time, the root node of the target text can be marked from the idle state to the matched state so that when processing the next text, the downstream matching can be started from the matched node. S130. Based on the root node hit result, determine the target processing method for the next text of the current text, and determine the hit result of the next text based on the target processing method.
[0047] It should be noted that if the number of characters in the target text is two or more, the processing result of the current character determines the processing method of the next character; if the number of characters in the target text is one, regardless of whether the current character hits the root node of the blacklist and whitelist state transition tree, the characters in the target text can be normal characters.
[0048] Among them, the target processing method is determined according to the hit result of the current text. For example, if the current text hits the root node of the blacklist state transition tree, the target processing method for the next text of the current text is to match the blacklist state transition tree and search the root node of the whitelist state transition tree, that is, match the leaf node that is the same as the next text from the leaf node associated with the root node, and at the same time, search the root node of the whitelist state transition tree for the text that is the same as the next text; if the current text hits the root node of the whitelist state transition tree, the target processing method for the next text of the current text is to match the whitelist state transition tree and search the root node of the blacklist state transition tree, that is, match the leaf node that is the same as the next text from the leaf node associated with the root node, and at the same time, search the root node of the blacklist state transition tree for the text that is the same as the next text; if the current text hits the root nodes of both the blacklist state transition tree and the whitelist state transition tree, the target processing method is to match the blacklist state transition tree and the whitelist state transition tree, that is, match the leaf node that is the same as the next text from the leaf node associated with the root node.
[0049] Specifically, the specific processing method for processing the next character of the current character can be determined based on the root node hit result of the current character, and then the next character and the characters after the next character can be processed based on the specific processing method to determine whether the consecutive characters hit the nodes on the same branch of the blacklist state transition tree, and / or whether the consecutive characters hit the nodes on the same branch of the whitelist state transition tree, and then determine whether it is a blacklist word or a whitelist word based on the hit result.
[0050] It should be noted that if the current text only hits root node 1 of the blacklist state transition tree, root node 1 will be marked from the idle state to the matched state; the next text is obtained, and it is determined whether the next text hits the whitelist root node, and whether the next text matches the text corresponding to the leaf node associated with root node 1. In other words, when the next text is obtained, if there are nodes in the matched state in both state trees, the search will start from the matched state node downward; if there are no nodes in the matched state, the matching will start from the root node.
[0051] S140: Determine the verification result of the target text according to the hit result of each character in the target text.
[0052] The verification result may include whether the target text includes whitelist words, whether the target text includes blacklist words, the specific content of the whitelist words, and / or the specific content of the blacklist words.
[0053] Specifically, based on the hit results of each word in the target text, it can be determined whether the target text includes whitelist words and / or blacklist words. At the same time, the blacklist and whitelist words can be output, or the target text can be sent to other terminal devices and the blacklist and whitelist words can be hidden and displayed.
[0054] It should also be noted that if the target text includes blacklist words and / or whitelist words, the blacklist words and / or whitelist words in the target text can be output and the user can be reminded.
[0055] The technical solution of the embodiment of the present invention is to input each character of the target text into a pre-trained black and white list state transition tree in sequence after obtaining the target text, obtain the node hit result corresponding to the current character, and determine the target processing method for the next character processing based on the node hit result, and then determine the existence of black and white list words in the target text based on the node hit result of each character, thereby improving the accuracy of target text detection while improving the text verification efficiency and resource utilization.
[0056] Based on the above technical solution, the method further includes: if the root node in the blacklist state transition tree and the whitelist state transition tree does not include the current text, then obtaining the next text of the current text and using the next text as the current text.
[0057] Specifically, if the current text hits neither the root node of the blacklist state transition tree nor the root node of the whitelist state transition tree, it means that the current text does not affect the attributes of the text associated with it. At this time, the next text of the current text can be obtained, and the next text can be used as the current text, and S110 to S140 can be re-executed, that is, to determine whether the current text hits the root node of the blacklist and whitelist state transition tree, and then determine the specific processing method of the next text of the current text.
[0058] On the basis of the above technical solutions, the method further includes: creating a blacklist state transition tree and a whitelist state transition tree respectively.
[0059] Specifically, a blacklist state transition tree is created based on the blacklist words in the blacklist word library, and at the same time, a whitelist state transition tree can be created based on the whitelist words in the whitelist word library.
[0060] In this embodiment, the blacklist state transition tree created based on the blacklist vocabulary in the blacklist vocabulary library can be found in Figure 2 For each blacklist word in the blacklist word library, the words with the same first letter are classified, and the first letter after classification is used as the root node of the blacklist state transition tree. For example, if there are two categories with the same first letter, the root nodes in the blacklist state transition tree can be marked as A1 and A2. For each word corresponding to each category, the words with the same second letter can be determined from the words associated with A1 and used as leaf nodes B1, B2...B associated with A1. n . Further, determine the leaf nodes B X Associated leaf node C x , and so on to determine each node of the blacklist state transition tree. In other words, the text corresponding to the node of the previous level shared by each node in the state transition tree is the same.
[0061] It should be noted that the depth of the blacklist state transition tree is consistent with the length of each word in the blacklist and whitelist vocabulary. For example, if the length of a blacklist word is 5 characters, the depth value corresponding to the branch in the state transition tree corresponding to the blacklist word is 5.
[0062] In this embodiment, the advantage of constructing a blacklist and whitelist state transition tree is that after obtaining the current text, the current text can be mutually verified based on the blacklist and whitelist state transition tree, thereby improving the technical effect of improving the efficiency and accuracy of blacklist vocabulary and whitelist vocabulary verification.
[0063] This technical solution determines whether a target text contains blacklist and whitelist terms based on a pre-created blacklist and whitelist state transition tree. Therefore, after obtaining the corresponding text, the target text can be verified against the blacklist and whitelist terms to determine whether the target text contains whitelist and / or blacklist terms, thereby improving the efficiency and accuracy of text verification.
[0064] Example 2
[0065] Figure 3 This is a flow chart of a text content verification method provided in Example 2 of the present invention. Based on the previous embodiment, the target processing method for the next character of the current character can be determined based on the root node hit result, and the hit result for the next character can be further optimized based on the target processing method. The specific implementation method can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the previous embodiment are not repeated here.
[0066] like Figure 3 As shown, the method includes:
[0067] S210: Obtain the first character of the target text and use the first character as the current text.
[0068] It should be noted that the target text is usually continuous, and the first character may be the first character in a sensitive word. Therefore, in order to improve the comprehensiveness of the verification of the target text, processing can be started from the first character of the target text, thereby improving the accuracy and comprehensiveness of the verification results.
[0069] S220: Determine a root node hit result corresponding to the current text based on a pre-built blacklist state transition tree and a pre-built whitelist state transition tree.
[0070] Among them, each node of the state transition tree is a corresponding word, that is, the state transition tree established based on the words in the blacklist vocabulary library, and the state transition tree established based on the words in the whitelist vocabulary library, each node is a corresponding word in the vocabulary.
[0071] Specifically, we can determine whether there is a target text that is consistent with the current text based on the text corresponding to each root node in the blacklist state transition tree and the whitelist state transition tree. If there is a target text that is the same as the current text, then the root node corresponding to the target text can be used as the root node hit result. Otherwise, the root node hit result is a miss. If it is a hit, the hit root node can be marked as a matched state, and the missed root node remains in an idle state.
[0072] S230: If the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree, obtain the next word of the current word and mark the target root node as a matched state.
[0073] Among them, the target root node is the root node that matches the current text. The root node can be the root node of the whitelist state transition tree or the root node of the blacklist state transition tree. Specifically, if the current text hits any target root node in the blacklist state transition tree and / or the whitelist state transition tree, the next text of the current text can be obtained. At the same time, this target root node can be marked as a matched state, so that when the next text is obtained, it can be started from the root node of the matched state to find whether the corresponding leaf node can be matched in the leaf node of the next level associated with it. If the state transition tree does not include a node in the matched state, when obtaining the next text, the search can be started from the root node.
[0074] S240. If the root node to be marked corresponding to the next character is matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, and the leaf node to be marked is matched from the next level associated with the target root node in the matched state, then the root node to be marked and the leaf node to be marked are used as the hit result of the next character, and the root node to be marked and the leaf node to be marked are marked as the matched state, so that when the next character of the next character is obtained, the hit result of the next character is determined starting from the node in each state transition tree that is marked as the matched state and has the largest depth value.
[0075] Among them, the target root node can be the root node of the blacklist state transition tree, or the root node of the whitelist state transition tree. Correspondingly, if the target root node belongs to the whitelist state transition tree, the other state transition tree refers to the blacklist state transition tree; if the target root node belongs to the blacklist state transition tree, the other state transition tree refers to the whitelist state transition tree. The root node to be marked can be understood as the target root node corresponding to the next character matched from the root node of the other state transition tree, and the root node is recorded as the root node to be marked. The leaf node to be marked is the node corresponding to the next character from the leaf node of the next level associated with the target root node. The matched node can be marked as a matched state, so that when the next character of the next character is obtained, that is, the third character, the node with the state of each node marked as a matched state and the node with the largest depth value can be started from the node to search downward to see whether the leaf node associated with it includes the third character. The state of each node can be determined in this way of mutual verification between black and white lists.
[0076] In this embodiment, if the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree, then the next word of the current word is obtained, and the target root node is marked as a matched state, it also includes: if the root node to be marked corresponding to the next word is not matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, or the leaf node to be marked is not matched from the next level associated with the target root node in the matched state, then the hit result of the next word is a miss, and the state of the target root node is updated from the matched state to the idle state, so that when the next word of the next word is obtained, matching starts from the root node of each state transition tree.
[0077] In actual applications, if the first character matches the target root node of the state transition tree, but the second character does not match the leaf node associated with the target root node, or the root node corresponding to the second character is not found in the state transition tree corresponding to the state transition tree to which the target root node belongs, the hit result for the second character is a miss. If the leaf node associated with the target root node is not hit, the state of the target root node can be updated from the matched state to the idle state.
[0078] It can be understood that if the next level associated with the leaf node with the largest depth value and in the matched state includes text corresponding to the next text, the node corresponding to the next text can be marked as a matched state; if it is detected that the next level associated with the leaf node with the largest depth value and in the matched state does not include text corresponding to the next text, the state of each node in the matched state and located on the same branch can be updated from the matched state to the idle state.
[0079] It should be noted that the mark of each node in the state transition tree in the initial state may be an idle state.
[0080] It should also be noted that when the third character is obtained, since all nodes in each state transition tree are in an idle state, the hit results corresponding to each character can be determined again starting from the root node. In other words, if there is a matched node in the matched state in the state transition tree, the node identical to the next character can be determined from the nodes in the next layer associated with the matched node. If there is no matched node in the state transition tree, when the next character is obtained, matching starts from the root node. Since the characters need to be entered into the blacklist and whitelist state transition tree after they are obtained, the blacklist and whitelist vocabulary in the target text can be determined according to the above principles.
[0081] On the basis of the above technical solutions, when the next character of the next character is obtained, the method also includes: for each state transition tree, if there is a node marked as a matched state, then starting from the target leaf node in the matched state and with the largest depth value, determine whether the next level associated with the target leaf node includes the next character; if so, mark the leaf node to be marked that is the same as the next character as a matched state; if not, update the matched state of the previous level of the target leaf node until the target root node to an idle state.
[0082] It can be understood that when the next text is obtained, if there is a node marked as a matched state in each state transition tree, then you can start from the target leaf node in the matched state and with the largest depth value, and determine whether the next level associated with the target leaf node includes the target text corresponding to the next text, that is, the leaf node to be marked. If it exists, the state of the leaf node to be marked can be marked as a matched state. Otherwise, you can start from the target leaf node and search upward for nodes in a matched state, and mark each node from a matched state to an unmatched state. Correspondingly, if there is no leaf node or root node in a matched state in the state transition tree, you can start from the root node to determine the target root node that matches the text.
[0083] In this embodiment, when obtaining the next text of the next text, the method also includes: if it is detected that the marks of each node are all in the idle state, determining whether the blacklist state transition tree and the whitelist state transition tree include the next text, and marking the target root node that is the same as the next text as the matched state.
[0084] It can be understood that when the first two characters, the first few characters, or a broken character occurs during the matching process, that is, a character does not match the node, the mark of each node may be idle. At this time, if the next character is obtained, S120 to S130 can be repeated, that is, determine whether there is a target character corresponding to the next character in the root node of each state transition tree. If so, the root node corresponding to the target character can be marked as idle. Otherwise, the status of all nodes is still idle, and the next character can be obtained.
[0085] S250: Determine the verification result of the target text according to the hit result of each character in the target text.
[0086] In this embodiment, based on the hit results of each character in the target text, the verification result of the target text can be determined as follows: if it is detected that the leaf node corresponding to the current character is the last leaf node of the blacklist state transition tree and is the leaf node to be marked in the whitelist state transition tree, then the next character of the current character is obtained, and the leaf node to be marked is marked as a matched state; if in the whitelist state transition tree, there is a target character corresponding to the next character in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the whitelist state transition tree are whitelist texts.
[0087] It should be noted that, in order to improve the text verification effect, this embodiment is mainly determined by combining the mutual verification of the blacklist and whitelist state transition tree.
[0088] Specifically, if it is detected that the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree, and the current text is the to-be-marked leaf node of the whitelist state transition tree, that is, the next level of the to-be-marked leaf node also includes a corresponding leaf node, then the next text of the current text can be obtained, and the to-be-marked leaf node can be marked as a matched state. After obtaining the next text, it can be determined that there is a target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value in the whitelist state transition tree and in the matched state, then the continuous characters in the target text that hit the nodes of the whitelist state transition tree are whitelist text.
[0089] In this embodiment, the verification result of the target text is determined based on the hit results of each character in the target text, including: if it is detected that the leaf node corresponding to the current character is the last leaf node of the whitelist state transition tree and is the leaf node to be marked of the whitelist state transition tree, then the next character of the current character is obtained, and the leaf node to be marked is marked as a matched state; if in the blacklist state transition tree, there is a target character corresponding to the next character in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist texts.
[0090] That is to say, if the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree, but the current text is not the leaf node of the last level in the corresponding to-be-marked node in the blacklist state transition tree, then obtain the next text of the current text, and determine that in the state transition tree, when there is a target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the consecutive characters in the target text that hit the blacklist state transition tree can be used as blacklist text.
[0091] In this embodiment, the verification result of the target text is determined according to the hit result of each character in the target text, including: if it is detected that the leaf node corresponding to the current character is the last leaf node of the blacklist state transition tree and is the to-be-marked leaf node of the whitelist state transition tree, then the next character of the current character is obtained, and the to-be-marked leaf node is marked as a matched state; if in the whitelist state transition tree, there is no target character corresponding to the next character in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist texts, and the to-be-marked leaf node in the whitelist state transition tree is marked as a matched state. Each node marked as matched is updated to an idle state; or, if it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree and is a leaf node to be marked in the blacklist state transition tree, the next text of the current text is obtained, and the leaf node to be marked is marked as matched; if in the whitelist state transition tree, there is no target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the whitelist state transition tree are whitelist texts, and each mark in the blacklist state transition tree is updated to an idle state.
[0092] It can be understood that, when it is detected that the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree, but not the last leaf node of the whitelist state transition tree, the next leaf node of the current text is obtained, and at the same time, the leaf node matched by the whitelist state transition tree is marked as matched. If there is no text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state in the whitelist state transition tree, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist texts, and each node in the blacklist and whitelist state transition trees is marked as matched and updated to an idle state. Alternatively, when it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree, but not the last leaf node of the blacklist state transition tree, the next leaf node of the current text is obtained, and at the same time, the leaf node matched by the blacklist state transition tree is marked as matched. If the text does not exist in the next-level leaf node associated with the leaf node with the largest depth value and in the matched state in the blacklist state transition tree, then the consecutive characters in the target text that hit the whitelist state transition tree are the whitelist text, and the marks of each node in the blacklist and whitelist state transition tree are updated from the matched state to the idle state.
[0093] In this embodiment, if the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree and is also the last leaf node of the whitelist state transition tree, the root node in the blacklist and whitelist state transition tree is determined to be hit for the first time. If the root node of the whitelist state transition tree is hit for the first time, the continuous text corresponding to each node hit by the whitelist state transition tree is determined to be the whitelist text; if the root node of the blacklist state transition tree is hit for the first time, the continuous text corresponding to each node hit by the blacklist state transition tree is determined to be the blacklist text.
[0094] The technical solution of the embodiment of the present invention determines the whitelist words and blacklist words in the target text by performing mutual verification on the current text through the blacklist and whitelist state transition tree, avoiding the problem in the prior art of needing to segment the text and then matching each word in the blacklist and whitelist vocabulary library according to the segmentation results, which has the problem of inaccurate segmentation and resulting in inaccurate verification results, as well as the technical problem of needing to verify the blacklist and whitelist separately and resulting in waste of resources, thereby achieving the technical effect of efficiently, conveniently and accurately determining sensitive words in the target text.
[0095] Example 3
[0096] As an alternative embodiment of the above embodiment, the text content verification method of this embodiment can be executed by a text content verification system, that is, a sensitive word detection system. In this embodiment, the sensitive word detection system includes: a text risk control system background, a polling searcher, a result output module, a state transition tree, and a detection controller. Figure 4 .
[0097] Among them, the text risk control system background is used to intercept the target text from the text information system background. For example, in the process of sending text, the text information system background obtains the target text, which can be sent after sensitive word detection through the risk control system (ie text content verification). The polling retriever is used to search each sentence in the target text, and under the command of the retrieval controller, each word in the text is input into the retrieval controller in turn. The retrieval controller is used to record the status of each node in the current black and white list state transition tree (matched state or idle state), and perform the next operation according to the status of each node. The state transition tree is constructed based on the vocabulary in the vocabulary library, and the schematic diagram of its construction can be seen in Figure 2 . The specific construction method can refer to the similar description of the first embodiment. For example, for the vocabulary in the vocabulary, take the starting word as the root node, traverse each word in each word, and insert it into the corresponding node in the state tree to construct a state transition tree. The result output module determines the risk control result of the current text based on the black and white list search results outputted in the end, or outputs the black and white list words or not only outputs the black and white list words but also outputs the corresponding prompt information.
[0098] Based on the above text content verification system, the specific implementation of the text content verification method can be:
[0099] The polling retriever obtains the first character in the target text and inputs the first target character into the retrieval controller; the retrieval controller includes a state transition tree and can determine whether the root node of the state transition tree includes the first character.
[0100] If the root nodes of the blacklist state transition tree and the whitelist state transition tree do not include the first word, the polling system obtains the next word to continue matching. If all the words do not match the corresponding root node, it means that the whitelist word and the blacklist word do not exist in the target text.
[0101] If the first character matches the root node of the blacklist state transition tree, the blacklist state transition tree matching process is started. At the same time, the first character is checked to see if it matches the root node of the whitelist state transition tree. If it also matches the root node of the whitelist state transition tree, the whitelist state transition tree matching process is also started. At the same time, the hit root node is updated from the idle state to the matched state.
[0102] If the first text only hits the root node of the whitelist state transition tree, see Figure 5 , then when the second word, the third word, and the last word are obtained in sequence, it is necessary to determine whether the word triggers the blacklist root node. If not, the output is based on whether it completely matches the whitelist word. That is, if the consecutive characters in the target text correspond to the nodes on the same branch of the whitelist state transition tree, it is a whitelist word, otherwise it is not a whitelist word.
[0103] If the blacklist root node is triggered in the process of obtaining the second word to the last word, the blacklist matching process is started synchronously, and the matched nodes are marked as matched. When the next word is obtained, the next word can be searched downward from the deepest node in the matched state to see if it hits the associated node. Of course, if it does not match, the matched state of each node marked as matched in the upward row starting from this node will be cancelled or updated to the idle state. It should be noted that if the first word only hits the root node of the blacklist state transition tree, the specific implementation method is the same as above, and only the whitelist word is modified to the blacklist word.
[0104] If the first character hits the root node of the blacklist and whitelist state transition tree at the same time, see Figure 6 , or first hit the root node of the blacklist state transition tree, and then in the subsequent matching process of each word, a word triggers a hit on the root node of the whitelist. In this case, it is necessary to determine whether the next node of the current node in the blacklist state transition tree is matched. If the blacklist word matches, but the whitelist word can continue to match, then the blacklist word is temporarily retained. If the whitelist word is finally matched, the temporarily stored blacklist word is discarded and the whitelist word is output. Otherwise, the newly stored blacklist word is output.
[0105] After matching each character, the final output result can be determined, and the output result may include information such as corresponding blacklist words and whitelist words.
[0106] The technical solution of the embodiment of the present invention is to input each character of the target text into a pre-trained black and white list state transition tree in sequence after obtaining the target text, obtain the node hit result corresponding to the current character, and determine the target processing method for the next character processing based on the node hit result, and then determine the existence of black and white list words in the target text based on the node hit result of each character, thereby improving the accuracy of target text detection while improving the text verification efficiency and resource utilization.
[0107] Example 4
[0108] Figure 7This is a structural diagram of a text content verification device provided in Example 5 of the present invention, which includes: a text extraction module 310, a root node matching module 320, a target processing method determination module 330 and a verification result determination module 340.
[0109] Among them, the text extraction module 310 is used to obtain the first character of the target text and use the first character as the current text; the root node matching module 320 is used to determine the root node hit result corresponding to the current text based on the pre-built blacklist state transition tree and whitelist state transition tree; the target processing method determination module 330 is used to determine the target processing method of the next text of the current text based on the root node hit result, and determine the hit result of the next text based on the target processing method; the verification result determination module 340 is used to determine the verification result of the target text according to the hit results of each text in the target text.
[0110] Based on the above technical solution, the root node matching module is also used to determine the target root node that matches the current text according to the target text corresponding to the root node in the blacklist state transition tree and the whitelist state transition tree, and use the target root node as the root node hit result of the current text.
[0111] Based on the above technical solutions, the device also includes: a text acquisition module, which is also used to: if the root node in the blacklist state transition tree and the whitelist state transition tree does not include the current text, then obtain the next text of the current text and use the next text as the current text.
[0112] Based on the above technical solutions, the target processing method determination module includes:
[0113] A state marking unit, configured to obtain the next character of the current character and mark the target root node as a matched state if the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree;
[0114] A child node determination unit is used to, if a root node to be marked corresponding to the next character is matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, and a leaf node to be marked is matched from the next level associated with the target root node of the matched state, take the root node to be marked and the leaf node to be marked as the hit result of the next character, and mark the root node to be marked and the leaf node to be marked as a matched state, so that when the next character of the next character is obtained, the hit result of the next character is determined starting from the node in each state transition tree whose node is marked as a matched state and has the largest depth value.
[0115] On the basis of the above-mentioned technical solutions, the state marking unit is used to obtain the next word of the current word if the hit result of the root node is any target root node in the blacklist state transition tree or the whitelist state transition tree, and mark the target root node as a matched state. It is also used to: if the root node to be marked corresponding to the next word is not matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, or the leaf node to be marked is not matched from the next level associated with the target root node in the matched state, the hit result of the next word is a miss, and the state of the target root node is updated from the matched state to the idle state, so that when obtaining the next word of the next word, matching starts from the root node of each state transition tree.
[0116] On the basis of the above technical solutions, when obtaining the next character of the next character, the device further includes:
[0117] a node judgment module for determining, for each state transition tree, if a node is marked as being in a matched state, whether the next word is included in the next level associated with the target leaf node that is in the matched state and has the largest depth value;
[0118] If so, the leaf node to be marked that is the same as the next character is marked as a matched state; if not, the matched state of the previous level of the target leaf node until the target root node is updated to an idle state.
[0119] Based on the above technical solutions, the device also includes: a matching module, which is used to determine whether the blacklist state transition tree and the whitelist state transition tree include the next text if it is detected that the marks of each node are all in the idle state, and mark the target root node that is the same as the next text as a matched state.
[0120] Based on the above technical solutions, the verification result determination module includes:
[0121] a child leaf node determination unit, configured to obtain the next character of the current character and mark the leaf node to be marked as a matched state if it is detected that the leaf node corresponding to the current character is the last leaf node of the blacklist state transition tree and is a leaf node to be marked of the whitelist state transition tree;
[0122] A whitelist determination unit is used to determine that if, in the whitelist state transition tree, a target text corresponding to the next text exists in a leaf node of the next level associated with a leaf node with the largest depth value and in a matched state, then the continuous characters in the target text that hit the whitelist state transition tree are whitelist texts.
[0123] Based on the above technical solutions, the verification result determination module includes:
[0124] If it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree and is the leaf node to be marked of the whitelist state transition tree, then the next text of the current text is obtained, and the leaf node to be marked is marked as a matched state; if in the blacklist state transition tree, there is a target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist text.
[0125] Based on the above technical solutions, the verification result determination module includes:
[0126] If it is detected that the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree and is a leaf node to be marked in the whitelist state transition tree, the next text of the current text is obtained, and the leaf node to be marked is marked as a matched state; if in the whitelist state transition tree, there is no target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist texts, and each mark in the whitelist state transition tree as a matched state is updated to an idle state; or,
[0127] If it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree and is a to-be-marked leaf node of the blacklist state transition tree, then the next text of the current text is obtained, and the to-be-marked leaf node is marked as a matched state;
[0128] If in the whitelist state transition tree, there is no target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the consecutive characters in the target text that hit the whitelist state transition tree are whitelist text, and each mark in the blacklist state transition tree as the matched state is updated to the idle state.
[0129] The technical solution of the embodiment of the present invention is to input each character of the target text into a pre-trained black and white list state transition tree in sequence after obtaining the target text, obtain the node hit result corresponding to the current character, and determine the target processing method for the next character processing based on the node hit result, and then determine the existence of black and white list words in the target text based on the node hit result of each character, thereby improving the accuracy of target text detection while improving the text verification efficiency and resource utilization.
[0130] The text content verification device provided by the embodiment of the present invention can execute the text content verification method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.
[0131] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the embodiments of the present invention.
[0132] Example 5
[0133] Figure 8 This is a structural diagram of an electronic device provided in Example 4 of the present invention. Figure 8 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 8 The electronic device 40 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention.
[0134] like Figure 8 As shown, electronic device 40 is a general-purpose computing device. Components of electronic device 40 may include, but are not limited to, one or more processors or processing units 401, system memory 402, and a bus 403 connecting various system components (including system memory 402 and processing unit 401).
[0135] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.
[0136] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.
[0137] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 8 Not shown, often called a "hard drive"). Although Figure 8 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.
[0138] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402. Such program modules 407 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.
[0139] The electronic device 40 may also communicate with one or more external devices 409 (e.g., keyboard, pointing device, display 410, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 40, and / or any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 8Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0140] The processing unit 401 executes various functional applications and data processing by running programs stored in the system memory 402, such as implementing the text content verification method provided by the embodiment of the present invention.
[0141] Example 6
[0142] Embodiment 6 of the present invention further provides a storage medium comprising computer-executable instructions, wherein the computer-executable instructions are used to execute a text content verification method when executed by a computer processor.
[0143] The method includes:
[0144] Obtain the first character of the target text and use the first character as the current text;
[0145] Determine a root node hit result corresponding to the current text based on a pre-built blacklist state transition tree and a pre-built whitelist state transition tree;
[0146] Determine a target processing method for the next character of the current character based on the root node hit result, and determine a hit result for the next character based on the target processing method;
[0147] The verification result of the target text is determined according to the hit result of each character in the target text.
[0148] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.
[0149] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0150] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0151] The computer program code for performing the operations of the embodiments of the present invention can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0152] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.
Claims
1. A text content verification method, characterized in that: include: Obtain the first character of the target text and use the first character as the current text; Based on the target text corresponding to the root nodes in the pre-built blacklist state transition tree and whitelist state transition tree, determine the target root node that matches the current text, and use the target root node as the root node hit result of the current text; Determine a target processing method for the next character of the current character based on the root node hit result, and determine a hit result for the next character based on the target processing method; Determining a verification result of the target text according to the hit results of each character in the target text; The step of determining a target processing method for a next character of the current character based on the root node hit result, and determining a hit result for the next character based on the target processing method, includes: If the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree, the next character of the current character is obtained, and the target root node is marked as a matched state.
2. The method according to claim 1, characterized in that Also includes: If the root nodes in the blacklist state transition tree and the whitelist state transition tree do not include the current character, the next character of the current character is obtained, and the next character is used as the current character.
3. The method according to claim 1, characterized in that The step of determining a target processing method for a next character of the current character based on the root node hit result, and determining a hit result for the next character based on the target processing method, further includes: If a root node to be marked corresponding to the next character is matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, and a leaf node to be marked is matched from the next level associated with the target root node of the matched state, then the root node to be marked and the leaf node to be marked are used as the hit result of the next character, and the root node to be marked and the leaf node to be marked are marked as matched state, so that when the next character of the next character is obtained, the hit result of the next character is determined starting from the node in each state transition tree whose node is marked as matched state and has the largest depth value.
4. The method according to claim 1, wherein If the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree, then obtaining the next character of the current character and marking the target root node as a matched state, the method further includes: If the root node to be marked corresponding to the next character is not matched from another state transition tree corresponding to the state transition tree to which the target root node belongs, or the leaf node to be marked is not matched from the next level associated with the target root node of the matched state, then the hit result of the next character is a miss, and the state of the target root node is updated from the matched state to the idle state, so that when obtaining the next character of the next character, the matching starts from the root node of each state transition tree.
5. The method according to claim 3 or 4, characterized in that When obtaining the next character of the next character, the method further includes: For each state transition tree, if there is a node marked as a matched state, then starting from the target leaf node in the matched state and with the largest depth value, determine whether the next level associated with the target leaf node includes the next word; If yes, mark the leaf node to be marked that is the same as the next character as matched; If not, the matched state of the upper level of the target leaf node up to the target root node is updated to an idle state.
6. The method according to claim 1 or 4, characterized in that When obtaining the next character of the next character, the method further includes: If it is detected that the marks of each node are all in the idle state, it is determined whether the root nodes of the blacklist state transition tree and the whitelist state transition tree include the next character, and the target root node identical to the next character is marked as a matched state.
7. The method according to claim 1, characterized in that Determining the verification result of the target text according to the hit result of each character in the target text includes: If it is detected that the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree and is a to-be-marked leaf node of the whitelist state transition tree, then the next text of the current text is obtained, and the to-be-marked leaf node is marked as a matched state; If in the whitelist state transition tree, there is a target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the node of the whitelist state transition tree are the whitelist text.
8. The method according to claim 1, characterized in that Determining the verification result of the target text according to the hit result of each character in the target text includes: If it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree and is the leaf node to be marked of the whitelist state transition tree, then the next text of the current text is obtained, and the leaf node to be marked is marked as a matched state; if in the blacklist state transition tree, there is a target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist text.
9. The method according to claim 1, characterized in that Determining the verification result of the target text according to the hit result of each character in the target text includes: If it is detected that the leaf node corresponding to the current text is the last leaf node of the blacklist state transition tree and is a leaf node to be marked in the whitelist state transition tree, the next text of the current text is obtained, and the leaf node to be marked is marked as a matched state; if in the whitelist state transition tree, there is no target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the continuous characters in the target text that hit the blacklist state transition tree are blacklist texts, and each node in the whitelist state transition tree is marked as a matched state and updated to an idle state; or, If it is detected that the leaf node corresponding to the current text is the last leaf node of the whitelist state transition tree and is a to-be-marked leaf node of the blacklist state transition tree, then the next text of the current text is obtained, and the to-be-marked leaf node is marked as a matched state; If in the whitelist state transition tree, there is no target text corresponding to the next text in the leaf node of the next level associated with the leaf node with the largest depth value and in the matched state, then the consecutive characters in the target text that hit the whitelist state transition tree are whitelist text, and each mark in the blacklist state transition tree as the matched state is updated to the idle state.
10. A text content verification device, characterized in that: include: A text extraction module is used to obtain the first character of the target text and use the first character as the current text; A root node matching module, configured to determine a root node hit result corresponding to the current text based on a pre-built blacklist state transition tree and a pre-built whitelist state transition tree; a target processing mode determination module, configured to determine a target processing mode for a next character of the current character based on the root node hit result, and determine a hit result for the next character based on the target processing mode; A verification result determination module, configured to determine a verification result of the target text according to a hit result of each character in the target text; The root node matching module is further configured to determine a target root node that matches the current text based on the target text corresponding to the root nodes in the blacklist state transition tree and the whitelist state transition tree, and use the target root node as the root node hit result of the current text; The target processing mode determination module includes: A state marking unit is used to obtain the next word of the current word and mark the target root node as a matched state if the root node hit result is any target root node in the blacklist state transition tree or the whitelist state transition tree.
11. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the text content verification method as described in any one of claims 1 to 9.
12. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the text content verification method according to any one of claims 1 to 9 when executed by a computer processor.
Citation Information
Patent Citations
Sensitive word matching method and system
CN103617251A