Sensitive content detection method and apparatus, electronic device, and storage medium

Abstract

The application provides a sensitive content detection method and device, electronic equipment and storage medium, and relates to the technical field of information identification. The method comprises the following steps: deconstructing a to-be-detected text based on a disassembled character library and a component radical library to obtain a target deconstruction result; traversing each character in the target deconstruction result, and searching for a matching target sensitive word variant from a target prefix tree corresponding to a target scene; when the matching target sensitive word variant is searched, it is determined that sensitive content exists in the to-be-detected text; wherein the target prefix tree is established based on a sensitive word variant obtained by deconstructing a sensitive word library of the target scene based on the disassembled character library and the component radical library; the root node of the target prefix tree is an empty node, the child nodes of the root node are multiple, and each node path formed from each child node of the root node to a mark node with an end symbol identifier represents a sensitive word variant. The scheme of the application can effectively detect sensitive content in time.

Description

Technical Field

[0001] This application relates to the field of information recognition technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting sensitive content. Background Technology

[0002] The rapid development of the internet has facilitated the release, exchange, and sharing of information, leading to an increasingly rich and diverse range of information published on online platforms. This has inevitably resulted in the spread of a large amount of harmful information. In particular, sensitive content with negative connotations, such as pornography and violent content, seriously impacts the healthy development of the internet environment and online platforms. Therefore, effectively identifying and filtering sensitive content is of great significance for the supervision of online platforms and the healthy development of the online environment.

[0003] Among related technologies, natural language processing (NLP) techniques such as deep learning and large artificial intelligence models are used to identify and extract sensitive content in text. Although this method can effectively cover generalized semantic scenarios, it relies on a large number of training samples collected in the early stage. When a large amount of sensitive content emerges, especially when a large number of new sensitive content appears, it is difficult to cover it in time, resulting in the inability to identify sensitive content in a timely and effective manner. Summary of the Invention

[0004] This application provides a sensitive content detection method, apparatus, electronic device, and storage medium to solve the problem that sensitive content cannot be identified in a timely and effective manner in the prior art.

[0005] Firstly, this application provides a method for detecting sensitive content, including:

[0006] The text to be detected is deconstructed based on the character decomposition library and radical library to obtain the target deconstruction result;

[0007] Iterate through each character in the target deconstruction result and retrieve the matching target sensitive word variants from the target prefix tree corresponding to the target scene;

[0008] If a matching variant of the target sensitive word is found, it is determined that there is sensitive content in the text to be detected;

[0009] The target prefix tree is established based on the sensitive word variants obtained by deconstructing the sensitive word library of the target scene using the character decomposition library and the radical library. The root node of the target prefix tree is an empty node, and the root node has multiple child nodes. Each node path formed from each child node to the end of the marker node represents a sensitive word variant. The marker node is a node with an end marker.

[0010] Optionally, the step of deconstructing the text to be detected based on the character decomposition library and radical library to obtain the target deconstruction result includes:

[0011] The initial deconstruction result is obtained by deconstructing each character in the text to be detected based on the character decomposition library.

[0012] The radicals in the initial deconstruction result are determined based on the radical library;

[0013] The radicals and components in the initial deconstruction result are removed to obtain the target deconstruction result.

[0014] Optionally, the step of deconstructing each character in the text to be detected based on the character decomposition library to obtain an initial deconstruction result includes:

[0015] Identify non-text characters in the text to be detected and remove them from the text to obtain the target text to be detected; the non-text characters are characters other than Chinese characters and phonetic characters;

[0016] The target text to be detected is deconstructed based on the character decomposition library, and the deconstructed target text to be detected is determined as the initial deconstruction result.

[0017] Optional, sensitive content detection methods also include:

[0018] Obtain the sensitive word database for the target scenario;

[0019] Based on the character decomposition library, each character in the sensitive word library is deconstructed to obtain the sensitive word deconstruction result;

[0020] Based on the radical library, the radicals of sensitive words in the sensitive word deconstruction results are determined, and the radicals of sensitive words are removed from the sensitive word deconstruction results to obtain Chinese character variants;

[0021] The sensitive word variants are generated based on the Chinese character variants;

[0022] Using an empty node as the root node, traverse each of the sensitive word variants, store the characters of each sensitive word variant in character order into the non-root nodes of the prefix tree, and add the end marker to the node corresponding to the last character of each sensitive word variant to obtain the target prefix tree; each non-root node of the prefix tree stores one character.

[0023] Optionally, generating the sensitive word variant based on the Chinese character variant includes:

[0024] Obtain the pinyin of each sensitive word in the sensitive word database from the pinyin database;

[0025] By cross-combining the Chinese character variants and the corresponding pinyin of the sensitive words, pinyin variants are obtained;

[0026] The Chinese character variant and the Pinyin variant are identified as the sensitive word variants.

[0027] Optionally, the step of traversing each character in the target deconstruction result and retrieving matching target sensitive word variants from the target prefix tree corresponding to the target scene includes:

[0028] Simultaneously traverse each character in the text to be detected and each character in the target deconstruction result, and retrieve matching target sensitive word variants from the target prefix tree corresponding to the target scene;

[0029] The text characters include Chinese characters and phonetic characters.

[0030] Optionally, after determining that sensitive content exists in the text to be detected, the sensitive content detection method further includes:

[0031] Based on the correspondence between sensitive word variants and sensitive word prototypes, the target sensitive word prototype corresponding to the target sensitive word variant is determined.

[0032] Secondly, this application provides a sensitive content detection device, comprising:

[0033] The deconstruction module is used to deconstruct the text to be detected based on the character decomposition library and radical library to obtain the target deconstruction result;

[0034] The retrieval module is used to traverse each character in the target deconstruction result and retrieve matching target sensitive word variants from the target prefix tree corresponding to the target scene. The target prefix tree is built based on the sensitive word variants obtained after deconstructing the sensitive word library of the target scene using the character splitting library and the radical library. The root node of the target prefix tree is an empty node, and the root node has multiple child nodes. Each node path formed from each child node to the end of a marker node represents a sensitive word variant. The marker node is a node with an end-of-terminal identifier.

[0035] The determination module is used to determine that sensitive content exists in the text to be detected when a matching variant of the target sensitive word is retrieved.

[0036] Thirdly, this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to implement the steps of the sensitive content detection method as described in any of the first aspects above.

[0037] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the sensitive content detection method as described in any of the first aspects above.

[0038] The sensitive content detection method, apparatus, electronic device, and storage medium provided in this application first deconstruct the text to be detected based on a character decomposition library and a radical library to obtain the target deconstruction result. Then, each character in the target deconstruction result is traversed, and a matching target sensitive word variant is retrieved from the target prefix tree corresponding to the target scene. If a matching target sensitive word variant is found, it is determined that there is sensitive content in the text to be detected. By searching for target sensitive word variants in the target deconstruction result of the text to be detected in the target prefix tree, the detection of sensitive content is achieved. The target prefix tree is built based on the sensitive word library of the target scene after deconstructing the sensitive words using the character decomposition library and radical library. The root node is an empty node, and the root node has multiple child nodes. Each node path formed from each child node of the root node to the marker node with the end symbol represents a sensitive word variant. In this way, when a large amount of sensitive content emerges or a large amount of new sensitive content appears, it is only necessary to add the sensitive words corresponding to these sensitive contents to the corresponding sensitive word library, deconstruct them, and update the sensitive word variants determined based on the deconstruction results to the target prefix tree. The target prefix tree can then be used to detect sensitive content in a timely and effective manner. Attached Figure Description

[0039] Figure 1 A flowchart illustrating the sensitive content detection method provided in this application embodiment;

[0040] Figure 2 This is one of the structural diagrams of the target prefix tree provided in the embodiments of this application;

[0041] Figure 3 This is the second schematic diagram of the target prefix tree provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram of the sensitive content detection device provided in an embodiment of this application. Detailed Implementation

[0043] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A alone, A and B simultaneously, or B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c alone can mean: a alone, b alone, c alone, a combination of a and b, a combination of a and c, a combination of b and c, or a, b, and c. a, b, and c can be single or multiple. Furthermore, the terms "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance.

[0044] The terms “center,” “longitudinal,” “lateral,” “up,” “down,” “left,” “right,” “front,” and “rear,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0045] The rapid development of the internet has facilitated the release, exchange, and sharing of information, leading to an increasingly richer and more diverse range of information published on online platforms. For example, users can freely post information on forums, social media platforms, and freight forwarding platforms, inevitably resulting in the spread of a large amount of harmful information. In particular, sensitive content with negative connotations, such as pornography and violent content, seriously impacts the healthy development of the internet environment and online platforms. Therefore, effectively identifying and filtering sensitive content is of great significance for the supervision of online platforms and the healthy development of the online environment.

[0046] In related technologies, sensitive word databases can be established, and regular expression matching can be used to identify sensitive content in text. Alternatively, Natural Language Processing (NLP) techniques such as Hidden Markov Models (HMM), Conditional Random Fields (CRF), word vector models, text clustering, deep learning, or large-scale artificial intelligence models can be used to identify and extract sensitive content from text. Alternatively, multi-dimensional information such as user accounts can be combined for joint judgment. While these methods can effectively cover generalized semantic scenarios, they rely on acquiring large amounts of multi-dimensional data, and model training requires collecting a large number of training samples. The initial preparation time is long, and when a large amount of sensitive content emerges, especially when new sensitive content appears in large quantities, it is difficult to cover it in a timely manner. Furthermore, it is difficult to complete technical iteration and development and deployment in a short period of time, resulting in the inability to identify sensitive content in a timely and effective manner, thus leading to the proliferation of sensitive content.

[0047] In view of this, this application provides a sensitive content detection method. It can deconstruct a sensitive word library of a target scene based on a character decomposition library and a radical library, and then build a corresponding target prefix tree based on the sensitive word variants obtained from the deconstruction. For the text to be detected, it can be deconstructed based on the character decomposition library and the radical library. Then, the target sensitive word variants of the target deconstruction results of the text to be detected are searched in the target prefix tree. Whether a matching target sensitive word variant is found determines whether sensitive content exists in the text to be detected, thus realizing the detection of sensitive content. When a large amount of sensitive content emerges or a large number of new sensitive content appears, it is only necessary to add the sensitive words corresponding to these sensitive contents to the corresponding sensitive word library, deconstruct them, and update them in the target prefix tree. The target prefix tree can then be used to detect sensitive content in a timely and effective manner. Moreover, it can also effectively identify variant words such as homonyms of sensitive words.

[0048] The sensitive content detection method provided in this application can be applied to electronic devices or to sensitive content detection devices installed in such electronic devices. These sensitive content detection devices can be implemented through software, hardware, or a combination of both. The electronic devices may include at least one of the following, but are not limited to: servers, mobile phones, computers, in-vehicle terminals, tablet computers, wearable devices, smart home devices, augmented reality (AR) devices, and virtual reality (VR) devices. Servers may include independent servers, virtual servers, or cluster servers.

[0049] The following example uses applications in electronic devices, combined with... Figures 1-3A specific description of the sensitive content detection method provided by the embodiments of this application is given.

[0050] Figure 1 The flowchart of the sensitive content detection method provided by the embodiments of this application is shown. Referring to Figure 1 As shown, the sensitive content detection method may include the following steps 110 to step 130.

[0051] Step 110: Deconstruct the text to be detected based on the character decomposition library and the radical library to obtain the target deconstruction result.

[0052] Among them, the character decomposition library and the radical library can be open source, and the electronic device can directly retrieve the open source character decomposition library and radical library.

[0053] For the text to be detected, the electronic device can use the character decomposition library and the radical library to perform Chinese character deconstruction on the text to be detected, retaining the main part of the Chinese characters and the phonetic characters (such as pinyin) for phonetic representation, to obtain the target deconstruction result.

[0054] Specifically, step 110 of deconstructing the text to be detected based on the character decomposition library and the radical library to obtain the target deconstruction result can be implemented through the following steps 111 to step 113.

[0055] Step 111: Deconstruct each character in the text to be detected based on the character decomposition library to obtain the initial deconstruction result.

[0056] The character decomposition library is a software tool or database for analyzing Chinese character structures, radicals, and strokes, and can split Chinese characters into basic components through algorithms. For example, the character "ma" can be split into two parts, "shi" and "ma". In the text to be detected, one Chinese character is one character, and the electronic device can deconstruct each Chinese character in the text to be detected based on the character decomposition library to obtain the initial deconstruction result.

[0057] In one embodiment, the text to be detected may be a text that only includes Chinese characters, and the electronic device can deconstruct each Chinese character in the text to be detected based on the character decomposition library to obtain the initial deconstruction result.

[0058] )]In another embodiment, the text to be detected may be a text that only includes text characters, and the text characters may include Chinese characters and phonetic characters (such as pinyin) for phonetic representation. The electronic device can deconstruct each Chinese character in the text to be detected based on the character decomposition library, and determine the deconstructed Chinese character deconstruction result and the phonetic characters as the initial deconstruction result.

[0059] In yet another embodiment, considering that there may be some interference characters in the text to be detected, such as punctuation marks, numbers, and special characters, etc., these interference characters can be filtered out before character deconstruction, and then character deconstruction can be performed.

[0060] Specifically, each character in the text to be detected is deconstructed based on a character decomposition library to obtain an initial deconstruction result, which may include: identifying non-character characters in the text to be detected, and removing the non-character characters from the text to be detected to obtain a target text to be detected; deconstructing each Chinese character in the target text to be detected based on the character decomposition library, and determining the deconstructed target text to be detected as the initial deconstruction result. Among them, non-character characters are other characters except Chinese characters and phonetic characters.

[0061] Exemplarily, the electronic device can identify non-character characters in the text to be detected, and use the method of regular expressions to replace these non-character characters with empty strings to achieve the purpose of removing non-character characters from the text to be detected, and then determine the text to be detected after removing non-character characters as the target text to be detected.

[0062] Exemplarily, the electronic device can use template matching methods or trained character recognition models to identify non-character characters in the text to be detected. Among them, the character recognition model can be obtained by supervised training of an initial character recognition model based on sample characters and corresponding non-character character labels, and can be used to identify non-character characters. The initial character recognition model can be a basic neural network model such as a convolutional neural network, and the sample characters at least include sample non-character characters.

[0063] Step 112: Determine the radicals in the initial deconstruction result based on a radical library.

[0064] Among them, the radical library is used to store the radicals of Chinese characters. For example, the character "码" can be disassembled into two parts, "石" and "马", and "石" is the radical, which is stored in the radical library. The radical library can select an open-source radical library.

[0065] After the electronic device obtains the initial deconstruction result, it can determine the radicals in the initial deconstruction result based on the open-source radical library by using methods such as matching and comparison.

[0066] Step 113: Remove the radicals in the initial deconstruction result to obtain a target deconstruction result.

[0067] After the electronic device determines the radicals in the initial deconstruction result, it removes the radicals from the initial deconstruction result, retains the main characters in the initial deconstruction result, and determines the initial deconstruction result after removing the radicals as the target deconstruction result.

[0068] It can be understood that if the initial deconstruction result of a Chinese character is the prototype of the Chinese character or no radical is matched in the radical library, then the Chinese character itself is its main part, that is, a main character, and it is directly retained. For example, for the Chinese character "日", its deconstruction result is the prototype, and the prototype "日" is its main part and is directly retained.

[0069] Step 120: Traverse each character in the target deconstruction result and retrieve the matching target sensitive word variant from the target prefix tree corresponding to the target scenario.

[0070] Among them, the target prefix tree is established based on the variant of the sensitive word obtained by deconstructing the sensitive word library of the target scenario based on the character splitting library and the radical library. The root node of the target prefix tree is an empty node, and there are multiple child nodes of the root node. Each node path formed from each child node of the root node to the marked node represents a variant of the sensitive word. Each non-root node is used to store a character, and the marked node is a node with an end symbol identifier, and the end symbol identifier is used to identify the end of a variant of the sensitive word.

[0071] Exemplarily, sensitive content can be classified by scenario, and a corresponding sensitive word library can be established for each type of scenario, and the sensitive word library is used to store sensitive words corresponding to the scenario. The target scenario is a type of scenario in the scenario classification, which can include pornographic scenarios, violent tendency scenarios, or uncivilized language scenarios, etc. For example, assume that the target scenario is a pornographic scenario, then its corresponding sensitive word library is used to save sensitive words related to pornographic content.

[0072] The variant of the sensitive word can include the variant of the Chinese character, or include the variant of the Chinese character and the variant of the pinyin. Among them, the variant of the Chinese character is the main part of each Chinese character in the sensitive word after removing the radical. For example, assume that the sensitive word is "物理", the main part of "物" after removing the radical is "勿", and the main part of "理" after removing the radical is "里", then the corresponding variant of the Chinese character of this sensitive word can be "勿里". The variant of the pinyin is the cross combination of the variant of the Chinese character and the pinyin of the sensitive word. Similarly, taking the sensitive word "物理" as an example, its variant of the Chinese character is "勿里", and the pinyin is "wu li", then the variant of the pinyin can include "勿li" and "wu里", and can also include its full pinyin "wu li".

[0073] Exemplarily, an electronic device can build a target prefix tree through the following steps: obtain the sensitive word library of the target scenario; deconstruct each character in the sensitive word library of the target scenario based on the character decomposition library to obtain the sensitive word deconstruction result; determine the sensitive word radicals in the sensitive word deconstruction result based on the radical library, and remove the sensitive word radicals from the sensitive word deconstruction result to obtain Chinese character variants; generate sensitive word variants based on the Chinese character variants; use the empty node as the root node, traverse each sensitive word variant, store the characters of each sensitive word variant in the non-root nodes of the prefix tree in the character order, and add an end flag at the node corresponding to the last character of each sensitive word variant to obtain the target prefix tree; wherein, each non-root node of the prefix tree stores a character.

[0074] In one embodiment, generating sensitive word variants based on Chinese character variants may include: determining the Chinese character variants as sensitive word variants.

[0075] For example, assume that the sensitive word library of the target scenario includes sensitive words "physics", "destroy bridge", "waves", "block", and "obstacle". After deconstructing these sensitive words and removing the sensitive word radicals from the sensitive word deconstruction result, the Chinese character variants corresponding to each sensitive word as shown in Table 1 below can be obtained:

[0076] Table 1

[0077]

[0078] Based on Table 1, using the empty node as the root node, traverse each sensitive word variant in Table 1, that is, the Chinese character variant, store the characters of each sensitive word variant in the non-root nodes of the prefix tree in the character order, and add an end marker at the node corresponding to the last character of each sensitive word variant, and the target prefix tree as shown in Figure 2 can be obtained, where the square nodes represent the nodes with end markers added.

[0079] Combining Table 1 and Figure 2 , the root node of the target prefix tree is an empty node, each of the remaining nodes stores one character of the Chinese character variant, there are multiple child nodes of the root node, and end markers are added at the nodes where the last characters "li", "qiao", "shou", "dang", and "wu" of each sensitive word are located as marker nodes. Each node path formed from each child node of the root node to the marker node represents a Chinese character variant, that is, represents a sensitive word variant.

[0080] It can be understood that in the target prefix tree, each leaf node is a marked node, and each node path formed from each child node of the root node to the leaf node can represent a Chinese character variant. There may be other marked nodes on this node path. On this node path, the path from the child node of the root node to each marked node can represent a Chinese character variant. For example Figure 2 On the node path of "shu - dang - wu" in

[0081] Based on Figure 2 the target prefix tree shown, the electronic device can traverse each character in the target deconstruction result. For this character, starting from the root node of the target prefix tree, it retrieves whether there is a sensitive word variant character that matches this character among the child nodes of the root node. If no match is found, it continues to retrieve whether the next character in the target deconstruction result matches a certain child node of the root node; if a match is found, for example, if "pi" is matched, it moves to the next character in the target deconstruction result and continues to retrieve whether there is a sensitive word variant character that matches this next character among the child nodes of the "pi" node. If no match is found, it returns to the root node and continues to retrieve the next character in the target deconstruction result from the root node; if a child node of the "pi" node is matched, for example, if "shou" is matched and the "shou" node is a marked node with an end marker, then at this time a complete target sensitive word variant is matched, that is, the target sensitive word variant "pi shou" that matches is retrieved, and it can be determined that there is sensitive content in the text to be detected. Then it continues to traverse the remaining characters in the target deconstruction result until it matches a leaf node or the current traversed character is not in the child nodes of the current node, and then the search position of the target prefix tree is restored to the root node and the subsequent characters are continued to be traversed.

[0082] It can be understood that when a target marked node with an end marker is matched and the child nodes of this target marked node no longer match, it is determined that a complete target sensitive word variant is matched, and this target sensitive word variant is the sensitive word variant represented by the target node path from the matched target child node of the root node to the end of this matched target marked node. For example, if "shu" and "dang" are matched in the target prefix tree for the text to be detected and "dang" is a marked node, then it is determined that a complete target sensitive word variant "shu dang" is matched. Suppose after "dang" is matched, the next character in the target deconstruction result of the text to be detected is "wu", which matches the child node "wu" of "dang" and "wu" is a marked node, then it is determined that at this time a complete target sensitive word variant "shu dang wu" is matched again. Suppose after "dang" is matched, the next character in the target deconstruction result of the text to be detected is "shao", which matches all the child nodes of "dang" ( Figure 2If it does not match (there is only one child node "勿"), it is impossible to continue finding the variant of the sensitive word, the search position returns to the root node, and the text to be detected traverses backward from "勺".

[0083] In this way, by searching for the variant of the target sensitive word in the target prefix tree established based on Chinese character variants, compared with directly retrieving the prototype of the sensitive word, it can effectively cover the homomorphic variant forms of the sensitive word, avoiding the risk of missed detection caused by avoiding sensitive word detection through homomorphic variation of the sensitive word. For example, "遮挡" may be mutated into "蔗档" to avoid sensitive word detection.

[0084] In another embodiment, the Chinese character variants of the sensitive word and the pinyin of the sensitive word can be cross-combined to obtain pinyin variants, and then the Chinese character variants and pinyin variants are determined as the sensitive word variants corresponding to the sensitive word library in the target scenario.

[0085] Specifically, generating sensitive word variants based on Chinese character variants may include: obtaining the sensitive word pinyin corresponding to each sensitive word in the sensitive word library from the pinyin library; cross-combining the Chinese character variants and the corresponding sensitive word pinyin to obtain pinyin variants; and determining the Chinese character variants and pinyin variants as sensitive word variants.

[0086] For example, taking the sensitive word "物理" as an example, its Chinese character variant is "勿里", and the corresponding sensitive word pinyin is "wuli". Then, by cross-combining "勿里" and "wu li", pinyin variants "勿li" and "wu里" can be obtained. The cross-combination can be the interchange of the Chinese character variant character and the pinyin corresponding to the same sensitive word character, such as the interchange of "勿" and "wu" to form a new variant combination.

[0087] Exemplarily, the pinyin variants may also include the full pinyin of the sensitive word. For example, the pinyin variants of "物理" may include not only "勿li" and "wu里", but also "wu li".

[0088] Taking the sensitive word library in the target scenario including sensitive words "物理", "破坏桥", "波涛", and "遮挡" as an example, Table 2 shows the corresponding relationship between the sensitive words, their pinyins, Chinese character variants, and pinyin variants. It should be noted that for the convenience of description, only some combinations of the pinyin variants of the sensitive words in Table 2 are shown, and those skilled in the art can obtain all possible combinations according to the cross-combination principle provided in the embodiments of the present application (such as the cross-combination principle of the sensitive word "物理" exemplified above).

[0089] Table 2

[0090]

[0091] Based on the variants of sensitive words in Table 2, that is, Chinese character variants and pinyin variants, with the empty node as the root node, traverse each variant of the sensitive word in Table 2, store each character of each variant of the sensitive word in the non-root nodes of the prefix tree in the order of the characters, and add an end marker to the node corresponding to the last character of each variant of the sensitive word, then the target prefix tree as shown in Figure 3 can be obtained. Among them, the square nodes represent the nodes with end markers added.

[0092] It should be noted that for the convenience of附图展示和说明, Figure 3 the target prefix tree shown only shows some combinations of the pinyin variants of the sensitive words. Those skilled in the art can obtain all possible combinations according to the cross-combination principle provided in the embodiments of this application (such as the cross-combination principle of the sensitive word "physics" exemplified above) and construct them into the target prefix tree. For example, the target prefix tree can also include the full pinyin of the sensitive word, that is, the pinyin of all characters of the sensitive word.

[0093] Combined with Table 2 and Figure 3 as shown, the root node of the target prefix tree is an empty node, and each of the remaining nodes stores one character of the Chinese character variant or one character of the pinyin variant. There are multiple child nodes of the root node, and each node path formed from each child node of the root node to the leaf node represents a variant of the sensitive word.

[0094] Based on the Figure 3 target prefix tree shown, traverse each character in the target deconstruction result, and retrieve the matching target sensitive word variant from the target prefix tree corresponding to the target scenario, which may include: synchronously traverse the text characters in the text to be detected and each character in the target deconstruction result, and retrieve the matching target sensitive word variant from the target prefix tree corresponding to the target scenario; among them, the text characters include Chinese characters and phonetic characters.

[0095] Specifically, it can traverse the text characters in the text to be detected and each character in the target deconstruction result synchronously according to the index position order of the text characters in the text to be detected, and retrieve the matching target sensitive word variant from the target prefix tree corresponding to the target scenario.

[0096] For example, if the text to be detected is "The license plate is blocked by zhe", and the target deconstruction result obtained after deconstructing this text to be detected based on the character splitting library and the radical library is "car low skin zhe when", it can traverse "car - car", "plate - low", "by - skin", "z - z", "h - h", "e - e" and "block - when" in sequence according to the index position order. Among them, the representation like "block - when" means that "block" and "when" are traversed synchronously.

[0097] It should be noted that the part "附图展示和说明" in the original text seems to be incorrect or incomplete. It might be better to have the correct and complete information for a more accurate translation. Here, it is translated as literally as possible.For example, synchronously traversing each character in the text to be detected and each character in the target deconstruction result, and retrieving matching target sensitive word variants from the target prefix tree corresponding to the target scene, may include: synchronously traversing each character in the text to be detected and each character in the target deconstruction result, and searching along the search path starting from the root node of the target prefix tree corresponding to the target scene; during the search process, for each first target character traversed in the text to be detected, if the second target character synchronized with the first target character in the target deconstruction result does not find a matching first node in the target prefix tree, obtaining the target pinyin corresponding to the first target character; and determining whether a matching target pinyin can be retrieved when continuing the search along the current search path. The second node; if no matching second node is found, the search path is backed to the root node of the target prefix tree, and the search continues from the root node to find the next character corresponding to the first target character and the second target character; if a matching second node is found and the second node is a non-marked node, the search continues according to the current search path to find the next character corresponding to the first target character and the second target character; this search process is repeated until a matching node is found that is a target marked node and the child node of the target marked node does not match the next character to be searched or the target marked node is a leaf node. Then the sensitive word variant corresponding to the path from the root node to the end of the target marked node is determined as the matching target sensitive word variant.

[0098] For example, combining Figure 3, taking the text to be detected "车牌被zhe挡" as an example, synchronously traverse the first pair of synchronous characters "车-车". Starting from the root node (empty node) of the target prefix tree, check if there is a node in the children of the root node that matches the character "车" in the target deconstruction result. At this time, no match is found, so the pinyin "che" of the character "车" in the text to be detected can be obtained. Then, check if there is a node in the children of the root node that matches the first character "c" of this pinyin. No match is found either. Then, continue to traverse the next pair of synchronous characters "牌" and "卑" in the order of the index position. Still no "卑" is found, so the pinyin "pai" of "牌" is obtained for matching search. At this time, the "p" node is matched, but the next character "a" of the pinyin cannot be matched in the children of the "p" node, and the "p" node is not a marked node, which means that the character "牌" in the text to be detected is not a character of the sensitive word either. At this time, the search path is returned to the root node, and continue to search the next pair of synchronous characters "被" and "皮"; at this time, "皮" is matched in the children of the root node, and the "皮" node is a non-marked node, then continue to judge whether the next pair of synchronous characters "z" is in the children of "皮". No match is found, so the search path in the target prefix tree is returned to the root node, and start searching from the root node again. At this time, "z" can be matched in the children of the root node, "h" can be continuously matched in the children of "z", "e" can be matched in the children of "h", and "当" can be matched in the children of "e". At this time, "当" is a leaf-like marked node, so it can be determined that a complete variant of the sensitive word is matched at this time, that is, "zhe当", so it can be determined that there is sensitive content in the text to be detected. Every time a complete variant of the sensitive word is matched, and the node where the last character of this variant of the sensitive word is located is a leaf node or the children of the node where the last character is located will not be continuously matched, the search path of the target prefix tree is restored to the root node and continue to traverse the subsequent characters.

[0099] In this way, by establishing a target prefix tree that includes Chinese character variants and pinyin variants, and adopting the method of synchronously traversing the literal characters in the text to be detected and each character in the target deconstruction result during the retrieval process, compared with directly retrieving the prototype of the sensitive word or separately retrieving the characters in the target deconstruction result, it can not only cover the variant forms of the same type and homophonic type based on the sensitive word, but also avoid the risk of missed detection caused by using phonetic characters to mutate the sensitive word to avoid sensitive word detection, further improving the coverage rate and accuracy of sensitive content detection. For example, "遮挡" may mutate to "zhe挡" and escape the detection of sensitive words.

[0100] In practical applications, considering that the text to be detected may use homophonic sounds such as flat and retroflex sounds or front and back nasal sounds to avoid the detection of sensitive content, based on this, the detection of flat and retroflex sounds and / or front and back nasal sounds can be added during the synchronous traversal retrieval process.

[0101] For example, determining whether a matching second node can be retrieved when the target pinyin continues to be retrieved according to the current retrieval path may include: when the first character of the target pinyin matches the current retrieval node and is a flat or retroflex character, determining whether the second character of the target pinyin is in the children nodes of the current retrieval node and whether the children node characters of the current retrieval node contain the first target node character; if the second character is not in the children nodes of the current retrieval node and the children node characters of the current retrieval node contain the first target node character, continue to determine whether the second character and the remaining characters of the target pinyin can be retrieved to match the second node on the node path starting from the children nodes of the node where the first target node character is located according to the current retrieval path of the target prefix tree. Among them, the flat or retroflex characters include "c", "z", and "s", and the first target node character is "h".

[0102] For example, assume that there are sensitive word variants "同成勺" and "同cheng勺" of the sensitive word "同城约" in the target prefix tree, and the text to be detected includes the character "同层约", and the target deconstruction result after deconstructing "同层约" is "同云勺". During the retrieval process, synchronously traverse each character in the target deconstruction result "同云勺" and the "同层约" in the text to be detected. First, the variant "同" can be matched in the target prefix tree. The "云" in the target deconstruction result is not in the children nodes of the "同" node. At this time, obtain the pinyin "ceng" of "层", and perform character traversal retrieval on the pinyin "ceng". The first character "c" of this pinyin matches the current "c" node retrieved in the target prefix tree and is a flat or retroflex character, and the children nodes of the "c" node include the "h" node, and the second character "e" of the pinyin "ceng" is not in the children nodes of the "c" node. Then, at this time, the character "h" can be added between the first character and the second character of the pinyin "ceng" to match the "h" node, and then continue to retrieve the second character "e" of the pinyin and the subsequent characters from the children nodes of the "h" node in the target prefix tree to determine whether a matching node can be retrieved.

[0103] In this way, during the process of synchronously traversing the text characters in the text to be detected and each character in the target deconstruction result, by detecting the flat or retroflex characters, it is possible to cover the flat or retroflex variant forms of the sensitive words, effectively avoid the risk of missed detection caused by using the flat or retroflex to mutate the sensitive words to avoid sensitive word detection, and further improve the coverage rate of sensitive content detection.

[0104] Exemplarily, determining whether a matching second node can be retrieved when the target pinyin continues to be retrieved along the current retrieval path may include: when the last character of the target pinyin matches the current retrieval node, the last character of the target pinyin is a nasal character, and the current retrieval node is not a marked node, determining whether the sub-node characters of the current retrieval node contain a second target node character and whether the node where the second target node character is located contains a pinyin end symbol; if the sub-node characters of the current retrieval node contain the second target node character and the node where the second target node character is located contains a pinyin end symbol, it is determined that a matching second node can be retrieved when the target pinyin continues to be retrieved along the current retrieval path, and the nodes on the node path formed from the node matching the first character of the target pinyin to the node where the second target node character is located are determined as the matching second nodes. Herein, the pinyin end symbol is used to identify the end of a Chinese character pinyin, the nasal character is "n", and the second target node character is "g".

[0105] For example, assume that there are sensitive word variants "同成" and "同cheng" of the sensitive word "同城" in the target prefix tree, and the text to be detected includes the character "同尘", and the target deconstruction result after deconstructing "同尘" is "同土". During the retrieval process, by synchronously traversing each character in the target deconstruction result "同土" and "同尘" in the text to be detected, the variant "同" can be first matched in the target prefix tree. "土" in the target deconstruction result is not in the sub-nodes of the "同" node. At this time, the pinyin "chen" of "尘" is obtained, and the characters of the pinyin "chen" are traversed and retrieved. The last character of this pinyin is the nasal character "n", and the currently retrieved node "n" is not a marked node. The sub-node characters of this node "n" include "g", and the node corresponding to "g" contains a pinyin end symbol. It is determined that the pinyin "chen" matches a matching second node in the target prefix tree, that is, the node path formed by the matched nodes forms the pinyin "cheng", and it can be further determined that "同尘" in the text to be detected matches the sensitive word variant "同cheng".

[0106] In this way, during the process of synchronously traversing the literal characters in the text to be detected and each character in the target deconstruction result, by performing nasal sound detection, it is possible to cover the front-back nasal sound variant forms based on sensitive words, effectively avoiding the risk of missed detection caused by using front-back nasal sounds to mutate sensitive words to avoid sensitive word detection, and further improving the coverage rate of sensitive content detection.

[0107] It should be noted that the sensitive words exemplified in the embodiments of the present application are only for illustrating the concept and implementation principle of the present application, and do not represent real sensitive words in actual applications.

[0108] Step 130: If a matching variant of the target sensitive word is found, determine that there is sensitive content in the text to be detected.

[0109] If a matching variant of the target sensitive word is detected, it indicates that there is a sensitive word corresponding to the target scene in the text to be detected, and it can be determined that there is sensitive content in the text to be detected.

[0110] For example, an electronic device can determine whether a matching variant of the target sensitive word was retrieved during the retrieval process after iterating through all the characters in the target deconstruction result of the text to be detected.

[0111] The sensitive content detection method provided in this application first deconstructs the text to be detected based on a character decomposition library and a radical library to obtain the target deconstruction result. Then, it traverses each character in the target deconstruction result and retrieves matching target sensitive word variants from the target prefix tree corresponding to the target scene. If a matching target sensitive word variant is found, it is determined that sensitive content exists in the text to be detected. Sensitive content detection is achieved by searching for target sensitive word variants in the target deconstruction result of the text to be detected in the target prefix tree. The target prefix tree is built based on the sensitive word variants obtained after deconstructing the sensitive word library of the target scene using the character decomposition library and the radical library. Its root node is an empty node, and the root node has multiple child nodes. Each node path formed from each child node of the root node to the leaf node represents a sensitive word variant. Thus, when a large amount of sensitive content emerges or new sensitive content appears, it is only necessary to add the sensitive words corresponding to these sensitive contents to the corresponding sensitive word library, deconstruct them, and update the target prefix tree with the sensitive word variants obtained from the deconstruction result. This allows for timely and effective detection of sensitive content using the target prefix tree. Its retrieval time complexity is O(m), where m represents the number of characters in the text to be detected, and O(m) means that m is a constant multiple of m. It is fast in recognizing sensitive content.

[0112] Furthermore, the sensitive content detection method provided in this application does not require the collection of a large number of training samples; it only requires maintaining a sensitive word library and a target prefix tree. A small sample size is sufficient to efficiently cover sensitive words in the target scenario. Moreover, it eliminates the need for lengthy pre-processing time such as large-scale sample collection and model training, resulting in faster technology iteration and more efficient development and deployment.

[0113] In one embodiment of the sensitive content detection method based on the above embodiments, after determining that there is sensitive content in the text to be detected, the sensitive content detection method may further include the following step 140.

[0114] Step 140: Based on the correspondence between sensitive word variants and sensitive word prototypes, determine the target sensitive word prototype corresponding to the target sensitive word variant.

[0115] For example, when building the target prefix tree of the target scenario, a correspondence relationship between the sensitive word variants and their corresponding sensitive word prototypes can be established, and this correspondence relationship can be stored in a file or a database. After the electronic device determines that there is sensitive content in the text to be detected, based on the target sensitive word variant, it can search for the sensitive word prototype corresponding to the target sensitive word variant from the correspondence relationship between the sensitive word variants and the sensitive word prototypes, and obtain the target sensitive word prototype.

[0116] For example, if the retrieved target sensitive word variant is "庶当", then according to the correspondence relationship between the sensitive word variants and the sensitive word prototypes, it can be determined that the corresponding sensitive word prototype is "遮挡".

[0117] In this way, when there is sensitive content in the text to be detected, it can be recognized which specific sensitive words are included in the text to be detected, providing a basis for subsequent text filtering, sensitive word prompting, and other processing.

[0118] The sensitive content detection method provided by the embodiments of this application can be applied to the detection of sensitive words in sensitive content scenarios such as pornographic content, violent tendency content, and uncivil terms, and can also be applied to the sensitive content detection scenario of custom sensitive words. For example, on network platforms such as forums, social platforms, and freight transportation platforms, the platform management party can customize the sensitive words for sensitive content detection.

[0119] According to the sensitive content detection method provided by the embodiments of this application, through experiments on sudden pornographic cases on the test platform, it is evaluated that the recall rate of this method can reach 90%, and the precision rate can reach 89%.

[0120] The embodiments of this application also provide a sensitive content detection device. Figure 4 The structural schematic diagram of the sensitive content detection device provided by the embodiments of this application is shown. Refer to Figure 4 As shown, the sensitive content detection device may include:

[0121] A deconstruction module 410, configured to deconstruct the text to be detected based on the character decomposition library and the radical library, and obtain a target deconstruction result;

[0122] A retrieval module 420, configured to traverse each character in the target deconstruction result, and retrieve the matching target sensitive word variant from the target prefix tree corresponding to the target scenario; wherein, the target prefix tree is established based on the sensitive word variants obtained by deconstructing the sensitive word library of the target scenario based on the character decomposition library and the radical library; the root node of the target prefix tree is an empty node, there are multiple child nodes of the root node, and each node path formed from each child node of the root node to the marked node represents a sensitive word variant; wherein, the marked node is a node with an end symbol identifier.

[0123] The determination module 430 is used to determine that sensitive content exists in the text to be detected when a matching variant of the target sensitive word is retrieved.

[0124] In one embodiment, the deconstruction module 410 may include: a deconstruction unit, configured to deconstruct each character in the text to be detected based on a character decomposition library to obtain an initial deconstruction result; a determination unit, configured to determine the radicals in the initial deconstruction result based on a radical library; and a removal unit, configured to remove the radicals in the initial deconstruction result to obtain a target deconstruction result.

[0125] In one embodiment, the deconstruction unit is specifically used to: identify non-text characters in the text to be detected, and remove the non-text characters from the text to be detected to obtain the target text to be detected; non-text characters are characters other than Chinese characters and phonetic characters; deconstruct each Chinese character in the target text to be detected based on the character decomposition library, and determine the deconstructed target text to be detected as the initial deconstruction result.

[0126] In one embodiment, the sensitive content detection device may further include a target prefix tree construction module, which is used to: acquire a sensitive word library of the target scene; deconstruct each character in the sensitive word library based on a character decomposition library to obtain a sensitive word deconstruction result; determine the sensitive word radicals in the sensitive word deconstruction result based on a radical library, and remove the sensitive word radicals from the sensitive word deconstruction result to obtain Chinese character variants; generate sensitive word variants based on the Chinese character variants; traverse each sensitive word variant with an empty node as the root node, store the characters of each sensitive word variant in the non-root node of the prefix tree in character order, and add an end marker to the node corresponding to the last character of each sensitive word variant to obtain the target prefix tree; each non-root node of the prefix tree stores one character.

[0127] In one embodiment, when generating sensitive word variants based on Chinese character variants, the target prefix tree construction module can be specifically used to: obtain the sensitive word pinyin corresponding to each sensitive word in the sensitive word library from the pinyin library; cross-combine the Chinese character variants and the corresponding sensitive word pinyin to obtain pinyin variants; and determine the Chinese character variants and pinyin variants as sensitive word variants.

[0128] In one embodiment, the retrieval module 420 can be specifically used to: synchronously traverse each character in the text to be detected and each character in the target deconstruction result, and retrieve matching target sensitive word variants from the target prefix tree corresponding to the target scene; wherein, the text characters include Chinese characters and phonetic characters.

[0129] In one embodiment, the determining module 430 is further configured to: after determining that there is sensitive content in the text to be detected, determine the target sensitive word prototype corresponding to the target sensitive word variant based on the correspondence between the sensitive word variant and the sensitive word prototype.

[0130] The sensitive content detection device provided in this application embodiment has the same implementation principle and beneficial effects as the sensitive content detection method provided in the above embodiments, and will not be described again here.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0132] This application also provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps of the sensitive content detection method described in any of the above method embodiments, which will not be repeated here.

[0133] Based on the sensitive content detection method described in any of the above embodiments, this application also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, and an optical data storage device, etc. This storage medium stores computer instructions for executing the sensitive content detection method described in any of the above embodiments, which will not be elaborated further here.

[0134] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware to implement them. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0135] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations thereof that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

Claims

1. A method for detecting sensitive content, characterized in that, include: The text to be detected is deconstructed based on the character decomposition library and radical library to obtain the target deconstruction result; Iterate through each character in the target deconstruction result and retrieve the matching target sensitive word variants from the target prefix tree corresponding to the target scene; If a matching variant of the target sensitive word is found, it is determined that there is sensitive content in the text to be detected; The target prefix tree is established based on the sensitive word variants obtained by deconstructing the sensitive word library of the target scene using the character decomposition library and the radical library. The root node of the target prefix tree is an empty node, and the root node has multiple child nodes. Each node path formed from each child node to the end of the marker node represents a sensitive word variant. The marker node is a node with an end marker. The process of deconstructing the text to be detected based on character decomposition and radical libraries to obtain the target deconstruction result includes: The initial deconstruction result is obtained by deconstructing each character in the text to be detected based on the character decomposition library. The radicals in the initial deconstruction result are determined based on the radical library; The radicals and components in the initial deconstruction result are removed to obtain the target deconstruction result.

2. The sensitive content detection method according to claim 1, characterized in that, The initial deconstruction result, obtained by deconstructing each character in the text to be detected based on the character decomposition library, includes: Identify non-text characters in the text to be detected and remove them from the text to obtain the target text to be detected; the non-text characters are characters other than Chinese characters and phonetic characters; The target text to be detected is deconstructed based on the character decomposition library, and the deconstructed target text to be detected is determined as the initial deconstruction result.

3. The sensitive content detection method according to claim 1 or 2, characterized in that, Also includes: Obtain the sensitive word database for the target scenario; Based on the character decomposition library, each character in the sensitive word library is deconstructed to obtain the sensitive word deconstruction result; Based on the radical library, the radicals of sensitive words in the sensitive word deconstruction results are determined, and the radicals of sensitive words are removed from the sensitive word deconstruction results to obtain Chinese character variants; The sensitive word variants are generated based on the Chinese character variants; Using an empty node as the root node, traverse each of the sensitive word variants, store the characters of each sensitive word variant in character order into the non-root nodes of the prefix tree, and add the end marker to the node corresponding to the last character of each sensitive word variant to obtain the target prefix tree; each non-root node of the prefix tree stores one character.

4. The sensitive content detection method according to claim 3, characterized in that, The generation of the sensitive word variant based on the Chinese character variant includes: Obtain the pinyin of each sensitive word in the sensitive word database from the pinyin database; By cross-combining the Chinese character variants and the corresponding pinyin of the sensitive words, pinyin variants are obtained; The Chinese character variant and the Pinyin variant are identified as the sensitive word variants.

5. The sensitive content detection method according to claim 4, characterized in that, The process of traversing each character in the target deconstruction result and retrieving matching target sensitive word variants from the target prefix tree corresponding to the target scene includes: Simultaneously traverse each character in the text to be detected and each character in the target deconstruction result, and retrieve matching target sensitive word variants from the target prefix tree corresponding to the target scene; The text characters include Chinese characters and phonetic characters.

6. The sensitive content detection method according to claim 1 or 2, characterized in that, After determining that sensitive content exists in the text to be detected, the sensitive content detection method further includes: Based on the correspondence between sensitive word variants and sensitive word prototypes, the target sensitive word prototype corresponding to the target sensitive word variant is determined.

7. A sensitive content detection device, characterized in that, include: The deconstruction module is used to deconstruct the text to be detected based on the character decomposition library and radical library to obtain the target deconstruction result; The retrieval module is used to traverse each character in the target deconstruction result and retrieve matching target sensitive word variants from the target prefix tree corresponding to the target scene. The target prefix tree is built based on the sensitive word variants obtained after deconstructing the sensitive word library of the target scene using the character splitting library and the radical library. The root node of the target prefix tree is an empty node, and the root node has multiple child nodes. Each node path formed from each child node to the end of a marker node represents a sensitive word variant. The marker node is a node with an end-of-terminal identifier. The determination module is used to determine that sensitive content exists in the text to be detected when a matching variant of the target sensitive word is retrieved; The deconstruction module includes: The deconstruction unit is used to deconstruct each character in the text to be detected based on the character decomposition library to obtain the initial deconstruction result; The determining unit is used to determine the radicals in the initial deconstruction result based on the radical library; The removal unit is used to remove the radicals from the initial deconstruction result to obtain the target deconstruction result.

8. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the sensitive content detection method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the sensitive content detection method as described in any one of claims 1 to 6.

Images