Method and device for identifying text risks

Abstract

The embodiments of this specification provide a method and apparatus for identifying text risks. A specific implementation of the method includes: determining words in acquired text content that match keywords in a preset keyword set as risk keywords; determining a matching pattern between the risk keywords and the text content; determining a set of text risk elements for the text content based on the risk keywords and the matching pattern; and determining risk information for the text content based on the set of text risk elements and a pre-trained classifier.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly, to a method and apparatus for identifying text risks. Background Art

[0002] With the development of information technology, all kinds of information are increasingly permeating every aspect of people's lives. However, when browsing information, people often encounter information with negative impacts, such as violence, gambling, pornography, and fraud, which has a certain negative impact on people. Therefore, to avoid these negative impacts, it is necessary to review text content before displaying or providing it to users to determine whether it contains any risks. Summary of the Invention

[0003] The embodiments of this specification describe a method and apparatus for identifying text risks, which are used to identify risks in text content.

[0004] According to a first aspect, a method for identifying text risks is provided, comprising: taking words in the acquired text content that match keywords in a preset keyword set as risk keywords; determining a matching pattern of the above-mentioned risk keywords relative to the above-mentioned text content; determining a text risk element set of the above-mentioned text content based on the above-mentioned risk keywords and the above-mentioned matching pattern; and determining risk information of the above-mentioned text content based on the above-mentioned text risk element set and a pre-trained classifier.

[0005] In one embodiment, the above-mentioned keyword set includes basic keywords, and variant keywords generated based on the basic keywords and preset multiple text variant methods. The above-mentioned method also includes: in response to the above-mentioned risk keyword being a variant keyword, determining the target basic keyword and target text variant method corresponding to the above-mentioned risk keyword; and the above-mentioned determination of the text risk element set of the above-mentioned text content based on the above-mentioned risk keyword and the above-mentioned matching pattern, including: determining the text risk element set of the above-mentioned text content based on the above-mentioned target basic keyword, the above-mentioned target text variant method and the above-mentioned matching pattern.

[0006] In one embodiment, the above-mentioned risk keywords include at least two keywords, and the above-mentioned at least two keywords correspond to at least two target basic keywords and at least one target text variation mode; and the above-mentioned determination of the text risk element set of the above-mentioned text content based on the above-mentioned target basic keywords, the above-mentioned target text variation mode and the above-mentioned matching pattern includes: using the keyword combination composed of the above-mentioned at least two target basic keywords, the above-mentioned matching pattern, the above-mentioned at least one target text variation mode, and the above-mentioned at least two target basic keywords to constitute the text risk element set of the above-mentioned text content.

[0007] In one embodiment, the above-mentioned risk keywords include at least two keywords, and the above-mentioned at least two keywords are basic keywords; and the above-mentioned text risk element set of the above-mentioned text content is determined based on the above-mentioned risk keywords and the above-mentioned matching pattern, including: a keyword combination composed of the above-mentioned risk keywords, the above-mentioned matching pattern and the above-mentioned at least two keywords, constituting the text risk element set of the above-mentioned text content.

[0008] In one embodiment, the above-mentioned multiple text variation modes include sound variation, shape variation, traditional Chinese, character skipping, and character splitting.

[0009] In one embodiment, the above-mentioned classifier is a machine learning model, and the above-mentioned machine learning model is trained in the following manner: obtaining a training sample set, wherein the training sample includes a sample risk element set and risk category information corresponding to the sample risk element set; taking the sample risk element set of the training sample in the above-mentioned training sample set as input, and taking the risk category information corresponding to the input sample risk element set as the expected output, to train the above-mentioned machine learning model.

[0010] In one embodiment, the above-mentioned classifier is a classifier based on probability statistics. The above-mentioned classifier based on the above-mentioned text risk element set and pre-trained classifier determines the risk information of the above-mentioned text content, including: determining the risk information of the above-mentioned text content based on the probability calculation of each preset risk category according to the above-mentioned text risk element set.

[0011] In one embodiment, the above-mentioned classifier is a naive Bayes classifier, which determines the first probability of each risk category and the second probability of each risk element under each risk category through pre-training; and the above-mentioned classifier based on the above-mentioned text risk element set and pre-trained classifier determines the risk information of the above-mentioned text content, including: calculating the probability that the above-mentioned text risk element set belongs to each risk category based on the first probability and second probability determined by the above-mentioned naive Bayes classifier through training; and taking the risk category corresponding to the maximum probability calculated as the risk information of the above-mentioned text content.

[0012] In one embodiment, the above-mentioned naive Bayes classifier is pre-trained in the following manner: based on a preset training sample set, the first probability of each risk category and the second probability of each risk element under each risk category are statistically obtained, and the training samples in the above-mentioned training sample set include a sample risk element set and risk category information corresponding to the sample risk element set.

[0013] In one embodiment, the above-mentioned determination of the matching mode of the risk keywords relative to the text content includes: in response to determining that the text content contains more words than the risk keywords, determining that the matching mode of the risk keywords relative to the text content is inclusion matching; in response to determining that the text content contains the same words as the risk keywords, determining that the matching mode of the risk keywords relative to the text content is exact matching.

[0014] According to a second aspect, a device for identifying text risks is provided, including: a keyword determination unit, configured to use words in the acquired text content that match keywords in a preset keyword set as risk keywords; a pattern determination unit, configured to determine the matching pattern of the above-mentioned risk keywords relative to the above-mentioned text content; a set determination unit, configured to determine the text risk element set of the above-mentioned text content based on the above-mentioned risk keywords and the above-mentioned matching pattern; and a risk determination unit, configured to determine the risk information of the above-mentioned text content based on the above-mentioned text risk element set and a pre-trained classifier.

[0015] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in any implementation manner of the first aspect.

[0016] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, the method described in any implementation manner in the first aspect is implemented.

[0017] According to the method and device for identifying text risks provided in the embodiments of this specification, first, the words in the text content that match the keywords in the preset keyword set are used as risk keywords, and then the matching pattern of the risk keywords relative to the text content is determined, and the text risk element set of the text content is determined based on the risk keywords and the matching pattern. Finally, based on the text risk element set and a pre-trained classifier, the risk information of the text content is determined. In this way, risk identification based on the text risk element set is realized. Compared with risk identification using only keywords, risk identification based on the text risk element set can comprehensively consider the impact of various risk elements on the identification results, thereby making the identification results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram showing an application scenario in which the embodiments of this specification can be applied;

[0019] Figure 2 A flowchart of a method for identifying text risks according to one embodiment is shown;

[0020] Figure 3 A flowchart showing the process of determining the risk information of text content using a Naive Bayes classifier as an example is shown;

[0021] Figure 4 A schematic block diagram of a device for identifying text risks according to an embodiment is shown. Detailed implementation manners

[0022] The technical solutions provided in this specification will be further described in detail below in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and are not intended to limit the invention. Additionally, it should be noted that for the sake of description, only the parts related to the relevant invention are shown in the accompanying drawings. It should be noted that, without conflict, the embodiments of this specification and the features in the embodiments can be combined with each other.

[0023] As described above, before presenting or providing text to a user for browsing, it is necessary to review the text content to determine whether there are risks in the text content. If there are risks, the text content can be intercepted, warned, reported, etc. [[ID=十六]] [[ID=十七]]

[0024] For this purpose, the embodiments of this specification provide a method for identifying text risks to identify the risks of text content. Figure 1 A schematic diagram of an application scenario to which the embodiments of this specification can be applied is shown. As Figure 1 shown, user X wants to publish a text content "Fei^Ting game↑Fen find me, tea benefits" suspected of gambling risks through terminal device 101. Before publishing, the electronic device 102 responsible for identifying text risks can obtain the text content and match the text content with the keywords in the preset keyword set, and use the matching words in the text content as risk keywords.

[0025] Here, the keyword set may include keywords related to various risks. For example, keywords related to pornographic risks, keywords related to gambling risks, keywords related to fraud risks, keywords related to violence risks, and so on. In practice, in order to bypass the risk prevention and control system, lawbreakers often mutate non-compliant texts and replace some or all of the characters in the texts to bypass the risk prevention and control system without affecting reading and achieve illegal promotion. Taking the above text content "Fei^ting game ↑fen find me, tea welfare" as an example, user X replaced the non-compliant text "airship" with "Fei^ting", "score" with "↑fen", and "check welfare" with "tea welfare". In response to this situation, the correct form of basic keywords can be set in the keyword set, such as airship, score, check welfare, etc., and variant keywords generated by mutating based on the basic keywords, such as Fei^ting, ↑fen, tea welfare, etc. In this way, when determining the risk keywords, the words in the text content that are the same as the keywords in the keyword set can be used as risk keywords. In this example, "Fei^ting", "↑fen", and "tea welfare" are used as risk keywords. It is also possible to use the basic keywords corresponding to the words in the text content that are the same as the keywords in the keyword set as risk keywords. In this example, "airship", "score", and "check welfare" are used as risk keywords.

[0026] After that, the electronic device 102 can determine the matching mode of the risk keywords relative to the text content. In this example, in addition to the risk keywords, the text content "Fei^ting game ↑fen find me, tea welfare" also contains content such as "game" and "add me". Therefore, there are more words in the text content than the risk keywords, and thus the matching mode is inclusion matching. Then, the electronic device 102 can determine the text risk element set of the text content based on the risk keywords "airship, score, check welfare" and the matching mode "inclusion matching". For example, use the risk keywords and the matching mode to form the text risk element set of the text content. Here, a pre-trained classifier may be stored in the electronic device 102, and this classifier can be used to represent the corresponding relationship between the text risk element set and the risk information. In this way, the electronic device 102 can input the text risk element set of the text content into the classifier to obtain the risk information of the text content. Among them, the risk information may include the risk category of the text content. In this example, the risk information may include gambling risk.

[0027] Continue to refer to Figure 2 , Figure 2 shows a flowchart of a method for identifying text risks according to an embodiment. It can be understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. As Figure 2 shown, the method for identifying text risks may include the following steps:

[0028] Step 201: Use the words in the obtained text content that match the keywords in the preset keyword set as risk keywords.

[0029] In this embodiment, a keyword set can be preset in advance. The keyword set can include keywords related to various risks, such as keywords related to pornographic risks, keywords related to gambling risks, keywords related to fraud risks, and so on. As an example, the keywords in the keyword set can be obtained through various methods. For example, they can be crawled from websites suspected of various risks through web crawlers, or they can be manually set by technicians based on business requirements. Thus, for the obtained text content, first, the text content can be tokenized to obtain at least one token. Then, the at least one token is matched with the keywords in the keyword set, and the matching words are used as risk keywords.

[0030] Step 202: Determine the matching mode of the risk keyword with respect to the text content.

[0031] In this embodiment, the matching mode of the risk keyword obtained in step 201 with respect to the text content can also be determined. As an example, the matching mode can include exact match, containment match, and so on.

[0032] In some optional implementation manners, step 202 can be specifically performed as follows:

[0033] First, in response to determining that the words included in the text content are more than the risk keyword, it can be determined that the matching mode of the risk keyword with respect to the text content is a containment match. For example, assume the text content is "Airship game score increase". Among them, the risk keywords include "airship" and "score increase". Since the words included in "Airship game score increase" are more than the risk keywords "airship" and "score increase", therefore, it can be determined that the matching mode of "airship" and "score increase" with respect to "Airship game score increase" is a containment match.

[0034] Then, in response to determining that the words included in the text content are the same as the risk keyword, it is determined that the matching mode of the risk keyword with respect to the text content is an exact match. For example, assume the text content is "Check benefits". The risk keyword of this text content includes "Check benefits". Since the words included in the text content are the same as the risk keyword, therefore, it can be determined that the matching mode of the risk keyword "Check benefits" with respect to the text content "Check benefits" is an exact match. Through this implementation manner, the matching mode of the risk keyword with respect to the text content can be determined.

[0035] Step 203: Based on the risk keyword and the matching mode, determine the text risk element set of the text content.

[0036] In this embodiment, based on the risk keywords obtained in step 201 and the matching patterns determined in step 202, a set of text risk elements of the text content can be determined. For example, each risk keyword and matching pattern can be used as a text risk element respectively, so as to obtain a set of text risk elements.

[0037] In some implementation manners, the pre-set keyword set may include basic keywords and variant keywords generated based on the basic keywords and a variety of pre-set text variant manners. Here, the basic keyword may refer to the keyword in the correct form.

[0038] In practice, in order to bypass the risk prevention and control system, lawbreakers often vary non-compliant texts to obtain the varied texts. Among them, the text variant manners may include sound variation, shape variation, traditional Chinese, skipping characters, splitting characters, etc. Among them, sound variation may refer to replacing some or all of the characters in the basic keyword with homophonic characters. Taking the basic keyword "check for benefits" as an example, the character "check" can be replaced with the homophonic character "tea", so as to obtain the variant keyword "tea benefits". Shape variation may refer to replacing some or all of the characters in the basic keyword with characters or graphics with similar shapes. Taking the basic keyword "score points" as an example, the character "up" can be replaced with the character "soil" with a similar shape or the graphic "↑", etc., so as to obtain variant keywords "soil points", "↑ points", etc. Traditional Chinese may refer to replacing some or all of the characters in the basic keyword with traditional Chinese characters. Taking the basic keyword "make a fortune" as an example, the character "make" can be replaced with the traditional Chinese character "發", so as to obtain the variant keyword "發大财". Skipping characters may refer to adding some symbols with no actual meaning and not affecting normal reading between multiple characters of the basic keyword. Taking "airship" as an example, symbols with no actual meaning such as ",", "……" can be added between "air" and "ship" to obtain variant keywords such as "air, ship", "air……ship". Splitting characters may refer to splitting some or all of the characters in the basic keyword. Taking "resign naked" as an example, the character "naked" can be split into "衤" and "fruit", so as to obtain the variant keyword "衤 fruit resign".

[0039] Based on this, the above method for identifying text risks may further include the following content: in response to the risk keyword being a variant keyword, determining the target basic keyword and the target text variant manner corresponding to the risk keyword.

[0040] In this implementation, when matching words in the acquired text content with words in the keyword set, if the matched word is not a basic keyword but a variant keyword, the basic keyword corresponding to the matched word can be further determined to be the target basic keyword, and the corresponding text variant mode can be the target text variant mode. If the matched word is a basic keyword, the word itself can be used as its corresponding target basic keyword, and there is no target text variant mode, that is, the target text variant mode is empty. Through this implementation, the target basic keyword and target text variant mode corresponding to the risk keyword can be determined, thereby mining more risk elements in the text content.

[0041] On this basis, step 203 can be specifically performed as follows: based on the target basic keywords, target text variations, and matching patterns, a set of text risk elements for the text content is determined. For example, the basic keywords contained in the text content, the target basic keywords corresponding to the variant keywords contained therein, the target text variations, and the matching patterns can be used as text risk elements, thereby obtaining a set of text risk elements.

[0042] In some implementations, the risk keywords matched in step 201 may include at least two keywords, both of which are base keywords. Based on this, step 203 may be further specifically performed as follows: a text risk element set for the text content is constructed based on the risk keywords, the matching pattern, and the keyword combination consisting of the at least two keywords.

[0043] In other implementations, the risk keywords matched in step 201 may include at least two keywords, and some or all of the at least two keywords are variant keywords. In this example, if a certain risk keyword is a basic keyword, the risk keyword itself can be used as its corresponding target basic keyword. At this time, the above-mentioned at least two keywords can correspond to at least two target basic keywords and at least one target text variant. On this basis, the above-mentioned step 203 can also be specifically performed as follows: a keyword combination consisting of the above-mentioned at least two target basic keywords, a matching pattern, at least one target text variant, and at least two target basic keywords is used to constitute a text risk element of the text content.

[0044] For example, assume the text content is "菲^艇游戏↑芬找我". The risk keywords of this text content include "菲^艇" and "↑芬". Among them, the target basic keyword corresponding to "菲^艇" is "飞艇", and the target text variation methods include sound variation and character skipping. The target basic keyword corresponding to "↑芬" is "上分", and the target text variation methods include shape variation and sound variation. The matching mode is inclusion matching. At this time, the text risk element set for this text content can be composed of "飞艇、上分", "包含匹配", "音变、形变、跳字", and "飞艇和上分".

[0045] Another example, assume the text content is "飞艇游戏↑芬找我". The risk keywords of this text content include "飞艇" and "↑芬". Among them, "飞艇" is the basic keyword, and the target basic keyword corresponding to it is itself, that is, "飞艇". The target basic keyword corresponding to "↑芬" is "上分", and the target text variation methods include shape variation and sound variation. The matching mode is inclusion matching. At this time, the text risk element set for this text content can be composed of "飞艇、上分", "包含匹配", "音变、形变", and "飞艇和上分".

[0046] Through this implementation method, a text risk element set containing various text risk elements can be obtained for the text content, thereby providing more reference elements for the determination of risk information and making the obtained risk information more accurate.

[0047] Step 204, based on the text risk element set and a pre-trained classifier, determine the risk information of the text content.

[0048] In this embodiment, based on a preset training sample set, a classifier can be pre-trained. This classifier can be used to represent the correspondence between the text risk element set and the risk information. In this way, based on the text risk element set determined in step 203 and the classifier, the risk information of the text content can be determined. For example, the text risk element set can be input into the classifier, and the classifier outputs the risk information. As an example, the risk information can include the risk category.

[0049] In practice, when it is determined that the text content has risks according to the risk information of the text content, the text content can also be further processed. For example, the text content can be reported, or a risk prompt message can be sent to the device that submitted the text content, etc.

[0050] In some optional implementation methods, the above classifier can be a machine learning model, and this machine learning model can be trained in the following way:

[0051] First, obtain the training sample set.

[0052] In this implementation, an electronic device used to train a machine learning model may obtain a training sample set. The training samples in the training sample set may include a sample risk element set and risk category information corresponding to the sample risk element set. The risk category information may be used to indicate the risk category of the text corresponding to the sample risk element set.

[0053] Then, the sample risk element set of the training samples in the training sample set is taken as input, and the risk category information corresponding to the input sample risk element set is taken as the expected output to train a machine learning model.

[0054] In this implementation, the above-mentioned machine learning model can be a classification model obtained based on various machine learning algorithms, such as linear regression, logistic regression algorithm, support vector machine (SVM), neural network, etc. For example, the classification model training process can be as follows: First, the sample risk element set in the training sample is input into the classification model to obtain the output category prediction result of the sample risk element set. Then, the difference between the output category prediction result obtained and the risk category information corresponding to the sample risk element set is calculated using a preset loss function. Then, based on the calculated difference, the network parameters of the classification model can be adjusted, and the training can be terminated when the preset training end conditions are met. For example, the preset training end conditions here may include but are not limited to at least one of the following: the training time exceeds the preset duration; the number of training times exceeds the preset number of times; the calculated difference is less than the preset difference threshold.

[0055] Here, various implementations can be used to adjust the parameters of the classification model based on the difference between the generated output category prediction result and the risk category information corresponding to the sample risk element set. For example, a BP (Back Propagation) algorithm or an SGD (Stochastic Gradient Descent) algorithm can be used to adjust the parameters of the classification model.

[0056] In some optional implementations, the classifier may be a classifier based on probability statistics, and step 204 may be specifically performed as follows: determining the risk information of the text content based on probability calculation of the text risk element set for each preset risk category.

[0057] In this implementation, the classifier can be a classifier based on probabilistic statistics, such as a naive Bayesian classifier, a semi-naive Bayesian classifier, a Bayesian network, or the like. The classifier can first calculate the probability of a set of text risk elements for each preset risk category, and then determine the risk information of the text content based on the calculation results. For example, the risk category corresponding to the maximum probability in the calculation results can be selected as the risk information.

[0058] Please continue to see Figure 3 , Figure 3 The figure shows a flow chart of determining the risk information of text content using the Naive Bayes classifier as an example. Figure 3 In the process shown, the Naive Bayes classifier can be pre-trained to determine the first probability of each risk category and the second probability of each risk element under each risk category. In the Naive Bayes algorithm, it is assumed that each risk element is independent. Specifically:

[0059] In step 301, business data can be preprocessed to create a training sample set. The training samples in this training sample set may include a set of sample risk elements and risk category information corresponding to the sample risk element set. For example, business data may refer to data generated in a business scenario requiring text risk identification. Preprocessing can be performed on each piece of business data, such as extracting a risk element set and labeling risk categories, to generate training samples.

[0060] Step 302: Based on the training sample set, the first probability of each risk category and the second probability of each risk element under each risk category are obtained. The above step 204 can be specifically performed as follows:

[0061] Step 303 : Calculate the probability that the text risk element set belongs to each risk category based on the first probability and the second probability determined by the training of the naive Bayes classifier.

[0062] Step 304: The risk category corresponding to the calculated maximum probability is used as the risk information of the text content.

[0063] For example, suppose the risk event category is defined as E, where e represents a single risk category and e h It can represent the h-th risk category; K represents the keyword, k i Indicates the i-th keyword; T indicates the matching pattern, t j Indicates the jth matching pattern; V indicates the text variant mode, v m Indicates the mth text variation; C indicates the keyword combination, c n represents the nth keyword combination; the training sample set M includes multiple training samples x, and the training sample x can include multiple elements such as k, t, v, c, etc.

[0064] In this way, based on the training sample set M, each risk category e h The probability estimate P(e h ), specifically:

[0065]

[0066] Among them, Card can represent the number of data, Card(M) can represent the number of training samples in the training sample set M, and Card(e h ) can represent event e h The number of data, that is, the risk category in the training sample set M is e h The number of training samples.

[0067] Based on the training sample set M, the probability of each risk element under each risk event category can also be estimated. Specifically:

[0068]

[0069] Among them, Card(e h , k i , t j , v m , c n ) can represent the risk category e h Including k i , t k , v m , c n The number of risk element data.

[0070] Calculated according to Bayes' formula:

[0071]

[0072] For all categories, P(k, t, v, c) is the same, so the final calculation is:

[0073]

[0074] In practice, when some risk elements are missing, Laplace correction can be used, which will not be described here.

[0075] Looking back at the above process, in the embodiments of this specification, the words in the text content that match the keywords in the preset keyword set are first used as risk keywords, and then the matching pattern of the risk keywords relative to the text content is determined, and the text risk element set of the text content is determined based on the risk keywords and the matching pattern. Finally, based on the text risk element set and the pre-trained classifier, the risk information of the text content is determined. In this way, risk identification based on the text risk element set is realized. Compared with risk identification using only keywords, risk identification based on the text risk element set can comprehensively consider the impact of various risk elements on the recognition results, thereby making the recognition results more accurate.

[0076] According to another embodiment, a device for identifying text risks is provided. The device for identifying text risks can be deployed in any device, platform, or device cluster with computing and processing capabilities.

[0077] Figure 4 FIG. 1 shows a schematic block diagram of an apparatus for identifying text risks according to an embodiment. Figure 4 As shown, the device 400 for identifying text risks includes: a keyword determination unit 401, configured to use words in the acquired text content that match keywords in a preset keyword set as risk keywords; a pattern determination unit 402, configured to determine the matching pattern of the above-mentioned risk keywords relative to the above-mentioned text content; a set determination unit 403, configured to determine the text risk element set of the above-mentioned text content based on the above-mentioned risk keywords and the above-mentioned matching pattern; and a risk determination unit 404, configured to determine the risk information of the above-mentioned text content based on the above-mentioned text risk element set and a pre-trained classifier.

[0078] In some optional implementations of this embodiment, the above-mentioned keyword set includes basic keywords, and variant keywords generated based on the basic keywords and preset multiple text variant modes. The above-mentioned device 400 also includes: a variant mode determination unit (not shown in the figure), configured to determine the target basic keywords and target text variant modes corresponding to the above-mentioned risk keywords in response to the above-mentioned risk keywords being variant keywords; and the above-mentioned set determination unit 403 is further configured to: determine the text risk element set of the above-mentioned text content based on the above-mentioned target basic keywords, the above-mentioned target text variant modes and the above-mentioned matching patterns.

[0079] In some optional implementations of this embodiment, the above-mentioned risk keywords include at least two keywords, and the above-mentioned at least two keywords correspond to at least two target basic keywords and at least one target text variation mode; and the above-mentioned set determination unit 403 is further configured to: use the keyword combination consisting of the above-mentioned at least two target basic keywords, the above-mentioned matching pattern, the above-mentioned at least one target text variation mode, and the above-mentioned at least two target basic keywords to constitute a text risk element set of the above-mentioned text content.

[0080] In some optional implementations of this embodiment, the above-mentioned risk keywords include at least two keywords, and the above-mentioned at least two keywords are basic keywords; and the above-mentioned set determination unit 403 is further configured to: constitute a text risk element set of the above-mentioned text content based on the keyword combination consisting of the above-mentioned risk keywords, the above-mentioned matching pattern and the above-mentioned at least two keywords.

[0081] In some optional implementations of this embodiment, the above-mentioned multiple text variation modes include sound change, shape change, traditional Chinese, skipping characters, and splitting characters.

[0082] In some optional implementations of this embodiment, the above-mentioned classifier is a machine learning model, and the above-mentioned machine learning model is trained in the following manner: obtaining a training sample set, wherein the training samples include a sample risk element set and risk category information corresponding to the sample risk element set; taking the sample risk element set of the training samples in the above-mentioned training sample set as input, and taking the risk category information corresponding to the input sample risk element set as the expected output, to train the above-mentioned machine learning model.

[0083] In some optional implementations of this embodiment, the above-mentioned classifier is a classifier based on probability statistics, and the above-mentioned risk determination unit 404 is further configured to: determine the risk information of the above-mentioned text content based on the probability calculation of the above-mentioned text risk element set for each preset risk category.

[0084] In some optional implementations of this embodiment, the above-mentioned classifier is a naive Bayes classifier, and the above-mentioned naive Bayes classifier determines the first probability of each risk category and the second probability of each risk element under each risk category through pre-training; and the above-mentioned risk determination unit 404 is further configured to: calculate the probability that the above-mentioned text risk element set belongs to each risk category based on the first probability and the second probability determined by the above-mentioned naive Bayes classifier after training; and use the risk category corresponding to the calculated maximum probability as the risk information of the above-mentioned text content.

[0085] In some optional implementations of this embodiment, the above-mentioned naive Bayes classifier is pre-trained in the following manner: based on a preset training sample set, the first probability of each risk category and the second probability of each risk element under each risk category are statistically obtained, and the training samples in the above-mentioned training sample set include a sample risk element set and risk category information corresponding to the sample risk element set.

[0086] In some optional implementations of this embodiment, the above-mentioned pattern determination unit 402 is further configured to: in response to determining that the above-mentioned text content contains more words than the above-mentioned risk keywords, determine that the matching mode of the above-mentioned risk keywords relative to the above-mentioned text content is inclusion match; in response to determining that the above-mentioned text content contains the same words as the above-mentioned risk keywords, determine that the matching mode of the above-mentioned risk keywords relative to the above-mentioned text content is exact match.

[0087] According to another embodiment, there is also provided a computer readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute Figure 2 The method described.

[0088] According to another embodiment, there is also provided a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, Figure 2 The method described.

[0089] Those skilled in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0090] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0091] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying text risks, comprising: Using words in the acquired text content that match keywords in the preset keyword set as risk keywords; Determining a matching mode of the risk keyword relative to the text content, wherein the matching mode includes inclusive matching and exact matching; Determining a text risk element set of the text content based on the risk keyword and the matching pattern, wherein the text risk element set includes the matching pattern; Based on the text risk element set and a pre-trained classifier, risk information of the text content is determined, wherein the risk information includes a risk category.

2. The method according to claim 1, wherein The keyword set includes basic keywords and variant keywords generated based on the basic keywords and multiple preset text variants. The method further includes: In response to the risk keyword being a variant keyword, determining a target basic keyword and a target text variant method corresponding to the risk keyword; and The determining of a set of text risk elements of the text content based on the risk keywords and the matching pattern includes: A set of text risk elements of the text content is determined based on the target basic keywords, the target text variation mode and the matching pattern.

3. The method according to claim 2, wherein: The risk keywords include at least two keywords, and the at least two keywords correspond to at least two target basic keywords and at least one target text variation mode; as well as The step of determining a set of text risk elements of the text content based on the target basic keywords, the target text variation mode, and the matching pattern includes: A text risk element set of the text content is formed using the at least two target basic keywords, the matching pattern, the at least one target text variation mode, and a keyword combination consisting of the at least two target basic keywords.

4. The method according to claim 2, wherein: The risk keywords include at least two keywords, and the at least two keywords are basic keywords; as well as The determining of a set of text risk elements of the text content based on the risk keyword and the matching pattern includes: A text risk element set of the text content is constructed based on the risk keyword, the matching pattern, and a keyword combination consisting of the at least two keywords.

5. The method according to claim 2, wherein: The multiple text variation modes include sound change, shape change, traditional Chinese, skipping characters, and splitting characters.

6. The method according to claim 1, wherein The classifier is a machine learning model, which is trained in the following way: Acquire a training sample set, wherein the training sample includes a sample risk element set and risk category information corresponding to the sample risk element set; The machine learning model is obtained by training by taking the sample risk element set of the training samples in the training sample set as input and the risk category information corresponding to the input sample risk element set as the expected output.

7. The method according to claim 1, wherein The classifier is a classifier based on probability statistics, and the risk information of the text content is determined based on the text risk element set and a pre-trained classifier, including: The risk information of the text content is determined based on the probability calculation of the text risk element set for each preset risk category.

8. The method according to claim 7, wherein: The classifier is a naive Bayes classifier, which is pre-trained to determine a first probability of each risk category and a second probability of each risk element under each risk category; And determining the risk information of the text content based on the text risk element set and a pre-trained classifier includes: Calculating the probability that the text risk element set belongs to each risk category based on the first probability and the second probability determined by the training of the naive Bayes classifier; The risk category corresponding to the calculated maximum probability is used as the risk information of the text content.

9. The method according to claim 8, wherein The naive Bayes classifier is pre-trained in the following manner: based on a preset training sample set, a first probability of each risk category and a second probability of each risk element under each risk category are statistically obtained, and the training samples in the training sample set include a sample risk element set and risk category information corresponding to the sample risk element set.

10. The method according to claim 1, wherein Determining a matching pattern of the risk keyword relative to the text content includes: In response to determining that the text content includes more words than the risk keyword, determining that a matching mode of the risk keyword relative to the text content is a containment match; In response to determining that the words included in the text content are the same as the risk keyword, it is determined that the matching mode of the risk keyword relative to the text content is an exact match.

11. A device for identifying text risks, comprising: a keyword determination unit configured to use words in the acquired text content that match keywords in a preset keyword set as risk keywords; a pattern determination unit configured to determine a matching pattern of the risk keyword relative to the text content, wherein the matching pattern includes inclusive matching and exact matching; a set determining unit configured to determine a text risk element set of the text content based on the risk keyword and the matching pattern, wherein the text risk element set includes the matching pattern; The risk determination unit is configured to determine risk information of the text content based on the text risk element set and a pre-trained classifier, wherein the risk information includes a risk category.

12. The device according to claim 11, wherein The keyword set includes basic keywords and variant keywords generated based on the basic keywords and a plurality of preset text variants. The device further includes: a variation mode determining unit configured to, in response to the risk keyword being a variation keyword, determine a target basic keyword and a target text variation mode corresponding to the risk keyword; and The set determination unit is further configured to: A set of text risk elements of the text content is determined based on the target basic keywords, the target text variation mode and the matching pattern.

13. The device according to claim 12, wherein The risk keywords include at least two keywords, and the at least two keywords correspond to at least two target basic keywords and at least one target text variation; and The set determination unit is further configured to: A text risk element set of the text content is formed using the at least two target basic keywords, the matching pattern, the at least one target text variation mode, and a keyword combination consisting of the at least two target basic keywords.

14. The device according to claim 12, wherein The risk keywords include at least two keywords, and the at least two keywords are basic keywords; and The set determination unit is further configured to: A text risk element set of the text content is constructed based on the risk keyword, the matching pattern, and a keyword combination consisting of the at least two keywords.

15. The device according to claim 12, wherein The multiple text variation modes include sound change, shape change, traditional Chinese, skipping characters, and splitting characters.

16. The device according to claim 11, wherein The classifier is a machine learning model, which is trained in the following way: Acquire a training sample set, wherein the training sample includes a sample risk element set and risk category information corresponding to the sample risk element set; The machine learning model is obtained by training by taking the sample risk element set of the training samples in the training sample set as input and the risk category information corresponding to the input sample risk element set as the expected output.

17. The device according to claim 11, wherein The classifier is a classifier based on probability statistics, and the risk determination unit is further configured to: The risk information of the text content is determined based on the probability calculation of the text risk element set for each preset risk category.

18. The device according to claim 17, wherein The classifier is a naive Bayes classifier, which is pre-trained to determine a first probability for each risk category and a second probability for each risk element under each risk category; and the risk determination unit is further configured to: Calculating the probability that the text risk element set belongs to each risk category based on the first probability and the second probability determined by the training of the naive Bayes classifier; The risk category corresponding to the calculated maximum probability is used as the risk information of the text content.

19. The device according to claim 18, wherein The naive Bayes classifier is pre-trained in the following manner: based on a preset training sample set, a first probability of each risk category and a second probability of each risk element under each risk category are statistically obtained, and the training samples in the training sample set include a sample risk element set and risk category information corresponding to the sample risk element set.

20. The device according to claim 11, wherein The mode determination unit is further configured to: In response to determining that the text content includes more words than the risk keyword, determining that a matching mode of the risk keyword relative to the text content is a containment match; In response to determining that the words included in the text content are the same as the risk keyword, it is determined that the matching mode of the risk keyword relative to the text content is an exact match.

21. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 10.

22. A computing device comprising a memory and a processor, characterized in that: The memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 10 is implemented.

Images