Text auditing method

Abstract

The invention discloses a text auditing method. The method comprises the following steps: formatting a to-be-audited text to obtain a formatted text; the to-be-audited text and the formatted text are input into a target text auditing model, weighted sensitive word weights of the to-be-audited text and weighted sensitive word weights of the formatted text are obtained, and the target text auditing model is used for matching sensitive words in the text and determining weights of the sensitive words in the text; and based on the weighted sensitive word weight of the to-be-audited text and the weighted sensitive word weight of the formatted text, auditing the to-be-audited text.

Description

Technical Field

[0001] The present application relates to the field of information security technology, and in particular to a text review method. Background Art

[0002] With the development of the Internet and the requirements of the network environment for content security, nickname compliance judgment and text review accuracy have become important issues. However, related technologies can only review specific words or phrases, and the recognition effect of interference from some special characters, split characters, or conversion between traditional and simplified Chinese is not ideal. At the same time, it is difficult to quickly find out whether the quality of the included sensitive words is qualified, resulting in a high misjudgment rate. The common machine learning models on the market have low accuracy for ultra-short texts and the models cannot be updated in real time. Summary of the invention

[0003] The embodiment of the present application provides a text review method to solve the problem of low accuracy of text recognition in related technologies.

[0004] In a first aspect, an embodiment of the present application provides a text review method, comprising: Format the text to be reviewed to obtain formatted text; The to-be-reviewed text and the formatted text are respectively input into the target text review model to obtain the weighted sensitive word weights of the to-be-reviewed text and the weighted sensitive word weights of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weights of the sensitive words in the text; The text to be reviewed is reviewed based on the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text.

[0005] In a second aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, which, when executed by a computer, implement the steps of the method described in the first aspect.

[0006] In an embodiment of the present application, the text to be reviewed is first formatted to obtain a formatted text, and then the text to be reviewed and the formatted text are respectively input into the target text review model to obtain the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weights of the sensitive words in the text. Finally, based on the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text, the text to be reviewed is reviewed. The embodiment of the present application determines sensitive words based on the text before formatting and the text after formatting, so that the determination result is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flowchart of a text review method provided by an embodiment of the present application; Figure 2 is a schematic diagram of a text review device provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0008] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0009] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0010] The following is combined with Figures 1 to 3 , a text review method provided by an embodiment of the present application is described in detail through specific embodiments and application scenarios.

[0011] like Figure 1 As shown, it is a flowchart of a text review method provided by an embodiment of the present application. Figure 1 As shown, the text review method may include: the contents shown in S101 to S103.

[0012] In S101, the text to be reviewed is formatted to obtain a formatted text.

[0013] Among them, formatting processing can standardize the text in the text to be reviewed, convert the symbols therein into standard characters, convert traditional characters into simplified characters, convert deformed characters such as split characters into standard characters, and remove special characters such as spaces, slashes, backslashes, special emoticons, etc., making the formatted text easier to review.

[0014] Among them, the formatting process of the text to be reviewed can be to generate a formatted text without generalized characters, traditional Chinese characters, variant characters, and special symbols by passing the text to be reviewed through a formatting rule engine.

[0015] The formatting rule engine can include multiple rule libraries. For example, it can include: a generalized character mapping rule library G = {g_1, g_2, ..., g_r}, which contains r specific rules for converting numbers and symbols into standard characters. For example, "one, ①,壹, Ⅰ" can all be converted to "1"; a traditional-simplified conversion rule library S = {s_1, s_2, ..., s_p}, which contains the relationships between p traditional Chinese characters and simplified Chinese characters; a defined variant character conversion rule library V = {v_1, v_2, ..., v_q}, which contains the relationships between q variant characters and standard characters. For example, the standard character for the variant "+" is "plus", and the standard character for "亻言" is "信"; a special character rule library P = {p_1, p_2, ..., p_q}, which contains q special characters that need to be removed, such as spaces, slashes, backslashes, special emojis, etc.

[0016] It should be noted that the text to be reviewed can be short texts or ultra-short texts such as nicknames to be reviewed, or it can be an introduction to be reviewed, or a product name to be reviewed, etc. Specifically, the embodiments of this application are not limited and shall be subject to actual applications.

[0017] Taking the nickname to be reviewed as an example, the nickname text to be reviewed passes through the formatting rule engine to generate a formatted text T without generalized characters, traditional Chinese characters, variant characters, and special symbols. 2 The nickname text to be reviewed is T. 1 .

[0018] The formatting rule engine matches the characters to be replaced that conform to the formatting rule definition according to the definitions in the rules, and then replaces them with the corresponding standard characters defined by the formatting rules. The formatting rule engine does not limit the specific implementation tools.

[0019] It should be noted that from the perspective of actual applications, it is not recommended to use a loop to match strings. When the number of formatting rules exceeds a certain number, such as 10,000, the response time of the formatting process will be very long. For example, it may exceed 10 ms. At this time, a high-performance rule engine can be used to replace the low-performance rule engine to improve efficiency.

[0020] In S102, the text to be reviewed and the formatted text are respectively input into the target text review model to obtain the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weights of the sensitive words in the text.

[0021] Among them, the target text review model can be a text review engine, and the text review engine can use one or a combination of regular expression matching, a dictionary tree-based matching algorithm, an inverted index-based matching algorithm, and a natural language processing-based matching algorithm. This embodiment does not make any specific limitations and is subject to actual application.

[0022] It should be noted that in actual application, when the number of sensitive word libraries is greater than a certain number, such as 1 million, and the matching throughput (Transactions Per Second, TPS) requirement is higher than a certain value, such as 500, a high-performance regular expression matching engine Hyperscan and a high-performance rule engine Drools can be used in combination. Based on the recommended text audit engine, the specific method is to use Hyperscan to match a sensitive word set, and use Drools to generate weighted sensitive word weights for the sensitive word set.

[0023] In one example, the specific process of using the text review engine to determine the violation weight is as follows: First, the nickname text to be reviewed is T 1 After the text review engine, T 1 The weighted sensitive word weight T 1 ’ .

[0024] Next, format the text T 2 After the text review engine, T 2 The weighted sensitive word weight T 2 ’ , and extract the matching sensitive word set T 3 . Match sensitive word set T 3 Contains the matched sensitive words, the part of speech of the sensitive words, the start and end positions of the sensitive words in the nickname, and the handling actions.

[0025] For example, the nickname to be reviewed is "Selling white nets and socks", and the formatted text is T 2 For "sell white tennis socks", match the sensitive word set T 3 It is ["sex website|noun|3|4|automatic review"].

[0026] In this embodiment, weights are determined for both the text to be reviewed and the formatted text. That is, the weights of sensitive words in the text before formatting and the weights of sensitive words in the text after formatting are determined. The weights of the two texts are determined before reviewing to improve the accuracy of text review.

[0027] In S103, the text to be reviewed is reviewed based on the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text.

[0028] In an embodiment of the present application, the text to be reviewed is first formatted to obtain a formatted text, and then the text to be reviewed and the formatted text are respectively input into the target text review model to obtain the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weights of the sensitive words in the text. Finally, based on the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text, the text to be reviewed is reviewed. The embodiment of the present application determines sensitive words based on the text before formatting and the text after formatting, so that the determination result is more accurate.

[0029] In a possible implementation of the present application, reviewing the text to be reviewed based on the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text may include: comparing the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text with the sensitive word weight thresholds, respectively; based on the comparison results, determining a review method for the text to be reviewed, the review methods including automatic review and manual review.

[0030] That is to say, the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text are compared with the sensitive word weight threshold to determine whether the weighted sensitive word weights of the two texts exceed a certain value, and then determine the review method and whether the text is an illegal text.

[0031] In a possible implementation of the present application, determining the review method of the text to be reviewed based on the comparison result may include: when the weighted sensitive word weight of the text to be reviewed is greater than or equal to the sensitive word weight threshold, and the weighted sensitive word weight of the formatted text is greater than or equal to the sensitive word weight threshold, extracting the named entity information of the formatted text, and matching the sensitive word information of the formatted text, the named entity information including the named entity word, the part of speech of the named entity word, and the position of the named entity word in the formatted text, and the sensitive word information including the matched sensitive word, the part of speech of the sensitive word, and the position of the sensitive word in the formatted text; based on whether the sensitive word information is included in the named entity information, determining the review method of the text to be reviewed.

[0032] That is, when T 1 ’ >Violation weight threshold T 2 ’ >Violation weight threshold, format text T 2 Use named entity recognition to extract named entity information and determine the matching sensitive word set T 3 Whether to include in the formatted text T 2 Named entity information.

[0033] The identification method of named entity information is not limited in this embodiment, and can be open source CRF, LSTM-CRF, BERT, NER, etc., subject to actual application. Named entity information may include named entity words, the part of speech of the named entity words, and the starting position and ending position of the named entity words in the nickname. The part of speech of the named entity words includes nouns, verbs, phrases, names, and unit names. The number of named entity words can be zero, one, or more.

[0034] The named entity recognition results are assembled into named entity set information and formatted into text T 2 The named entity set information is T 4 , the collection information can be in the form of JSON, array, etc.

[0035] For example, formatting text T 2 For "sell white tennis socks", the named entity set information T 4 is ["sell|verb|0|1", "white|noun|2|3", "tennis socks|noun|4|6", "sell white tennis socks|phrase|0|6"].

[0036] Determine the matching sensitive word set T 3 Whether it is included in the named entity set information is T 4 In the example, traversal comparison and intersection operation can be used to determine that the words, word parts of speech, word start positions, and word end positions are all the same at the same time as inclusion relationships.

[0037] In a possible implementation of the present application, determining the review method of the text to be reviewed based on whether the sensitive word information is included in the named entity information may include: when all sensitive words in the sensitive word information are included in the named entity information, determining that the review method of the text to be reviewed is automatic review, and determining that the text to be reviewed is an illegal text; storing the correspondence between the text to be reviewed and the review conclusion, and the correspondence between the formatted text and the review conclusion in an audit feature vector database, the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

[0038] That is to say, when all sensitive words in the sensitive word information are included in the named entity information, it can be determined that the review method of the to-be-reviewed text is automatic review, that is, when the matching sensitive word set T 3 The handling actions for all sensitive words in the file are all automatic review, and when the text to be reviewed is determined to be an illegal text, the review feature vector database can be updated, and two key-value pairs are added to the review feature vector database: the key is T 1V、 The value is the combination of the audit conclusion and the nickname T1, and the key is T 2V、 The value is the combination of the audit conclusion and the nickname T2.

[0039] For example, [0.2, 0.5, -0.1, ..., 0.6, 0.8]:{"nickname": "Selling white net socks","approval_result": 0}.

[0040] In a possible implementation of the present application, determining the review method for the text to be reviewed based on whether the sensitive word information is included in the named entity information may include: when the sensitive words in the sensitive word information are not included or not all included in the named entity information, determining the review method for the text to be reviewed as human review; and sending the text to be reviewed, the formatted text and the sensitive word information to the target person for review.

[0041] In other words, if the sensitive word information is not all included in the named entity information, it cannot be reviewed through automatic review, and manual intervention is required. That is, the review method of the text to be reviewed is human review, and the text to be reviewed, formatted text and sensitive word information are sent to the target person for review. For example, they are sent to experts for review.

[0042] In one example, the expert review process is to receive and parse the materials to be reviewed. The material information can be received using the http protocol or other protocols, subject to actual application. The material data format is not limited, and can be in JSON format or other formats, subject to actual application. The material information includes the nickname text to be reviewed T 1 , format text T 2 , match sensitive word set T 3 , vector retrieval result set R, text semantic vector T 1V , text semantic vector T 2V For the text to be reviewed in the above materials, such as the nickname text T 1 , format text T 2 Perform manual review by experts. Based on the results of manual review by experts, add a key T to the feature vector database for data that is inconsistent with the results of machine review. 1V、 The value is a key-value pair of the combination of the expert manual review conclusion and the nickname T1, and the key is T 2V、 The value is a key-value pair consisting of the expert's manual review conclusion and the nickname T2.

[0043] Among them, sensitive word review includes the following steps: The first step is to analyze and match the sensitive word set T 3 Keywords that need to be reviewed in T 3The keyword counts that appear in the set T are stored in a NoSQL database in the form of key-value pairs, with the keyword as the key and the counter as the value of the corresponding key. 3 Check the database for the corresponding key and count value for each keyword that needs to be reviewed. If it exists, the count value is increased by 1 through the atomic operation provided by the database; if it does not exist, a new key is added and the count value is initialized to 1. If the count value is greater than the count threshold that needs to be reviewed, it is determined that the keyword needs to be reviewed.

[0044] In the second step, the naive Bayes classifier is trained using the improved custom features to determine whether the keyword is prone to misjudgment and needs to be deleted, and the keyword weight is recalculated.

[0045] Collect a large amount of nickname data and audit result data, and establish a data set containing vocabulary, manual audit results, and related features. Pre-process keywords, fuse the feature value TF-IDF and keyword part-of-speech information as improved custom features, thereby providing richer semantic features in classification tasks. Fusion feature calculation formula: Perform weighted summation of the normalized part-of-speech feature value P'(k) and the TF-IDF feature value T(k) to obtain the fused feature value F(k): F(k)=α* T(k) +β* P'(k), where α and β represent the weights of TF-IDF and part-of-speech features, respectively.

[0046] The calculation process of the part-of-speech feature P'(k) is as follows: For the part-of-speech information P(k) of keyword k, calculate the part-of-speech feature value P'(k): P'(k) = encoding value * weight value The encoding value represents the part-of-speech encoding of keyword k, and the weight value represents the weight of the part-of-speech, which is used to indicate its importance to the classification task. The part-of-speech feature P'(k) is normalized to a range between 0 and 1. This can be achieved by using methods such as maximum and minimum normalization, but not limited to.

[0047] The third step is for experts to manually review keywords and deal with them. If the expert manually reviews and determines that the keyword is a fuzzy word, the corresponding key value will be deleted from the keyword library to be reviewed, and the weight of the sensitive word rule in the sensitive word rule library will be reduced to the keyword fusion feature weight value obtained by training the naive Bayes classifier using improved custom features in the previous step; if the expert manually reviews and determines that the keyword is non-sensitive, the corresponding key value will be deleted from the keyword library to be reviewed, and the sensitive word rule will be deleted from the sensitive word rule library.

[0048] In a possible implementation of the present application, determining the review method of the text to be reviewed based on the comparison result may include: when the weighted sensitive word weight of the text to be reviewed is less than the sensitive word weight threshold, and / or the weighted sensitive word weight of the formatted text is less than the sensitive word weight threshold, inputting the text to be reviewed and the formatted text into the target text vector model to obtain the text semantic vector of the text to be reviewed and the text semantic vector of the formatted text; determining a first similarity between the text semantic vector of the text to be reviewed and each vector feature in the review feature vector database, and a second similarity between the text semantic vector of the formatted text and each vector feature in the review feature vector database; when the first similarities are both less than or equal to the first similarity threshold, and the second similarities are both less than or equal to the second similarity threshold, determining that the review method of the text to be reviewed is automatic review, and determining that the text to be reviewed is normal text; storing the correspondence between the text to be reviewed and the review conclusion, and the correspondence between the formatted text and the review conclusion in the review feature vector database, the review feature vector database being used to store various illegal texts and normal texts, as well as the review conclusion.

[0049] That is, if the weighted sensitive word weight of the text to be reviewed is less than the sensitive word weight threshold, and / or the weighted sensitive word weight of the formatted text is less than the sensitive word weight threshold, then the nickname text to be reviewed T 1 , format text T 2 Use the text vector model to generate text semantic vectors, and search from the audit feature vector database to determine whether text with a similarity greater than the threshold can be retrieved. 1 Generate text semantic vector T using text vector model 1V , format text T 2 Generate text semantic vector T using text vector model 2V For the vector T to be queried 1V and T 2V , calculate their similarity scores with each key in the audit feature vector database and filter out those that exceed the set similarity threshold Sim(T 1V ,K i ) and Sim(T 2V , K i ), the corresponding value V i , similarity is added to the vector retrieval result set R, let R i0 For text content, R i1 To review the conclusion, R i2If the first similarity is less than or equal to the first similarity threshold, and the second similarity is less than or equal to the second similarity threshold, that is, the vector search result set R is empty, the text to be reviewed is determined to be normal text, and the correspondence between the text to be reviewed and the review conclusion, as well as the correspondence between the formatted text and the review conclusion are stored in the review feature vector database.

[0050] Among them, the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

[0051] Among them, the target text vector model can be the nickname text embedding extracted by using the optimized and improved Roberta model. The training and model optimization methods are as follows: In the Roberta model, the self-attention mechanism can effectively model global dependencies, but sometimes ignores some important local features. In the nickname short text classification scenario, these local features contain key information. In order to better capture these features, multiple convolution operations with different numbers of kernels can be added to the multi-head attention of the Roberta model, which can provide feature representations at different levels and scales, introduce local feature information, and fuse it with global features, thereby improving the model's abstraction and generalization capabilities for input sequences. This method is particularly important for short text classification tasks with noise or complex language structures.

[0052] For nickname short text prediction tasks, too many layers are usually not needed to capture the semantic information in the text. Therefore, by reducing the number of Transformer layers and the number of attention heads in the model, the computational cost can be reduced while maintaining the prediction accuracy. This makes the model more lightweight and efficient, and more suitable for deployment and application in resource-constrained environments.

[0053] The Roberta model is composed of multiple layers of Transformers, each of which can learn semantic information at different levels of abstraction. Generally speaking, the features of the lower layers are closer to the original text and contain richer local context information, which helps to capture some fine-grained features in short texts. The low-level features of the model are of great significance for short text embedding. In the Roberta model, the first layer of Transformer contains more word-level information, while the last layer of Transformer contains more sentence-level semantic information. Therefore, integrating the two can consider both word-level and sentence-level information at the same time, thereby improving the model's ability to represent short texts. After reducing the number of Transformer layers, a feedforward neural network is introduced to fuse the embedding representations of the first layer of Transformer and the last layer of Transformer. The two are combined through appropriate weight parameters α and β to obtain a fusion model that contains both low-level and high-level semantic information, thereby more reasonably representing the input text.

[0054] In summary, by introducing local features and optimizing the model structure, the accuracy of the Roberta model in the nickname short text classification task can be improved, and the computational cost can be reduced, making it more suitable for application in practical scenarios.

[0055] The model training data mainly consists of the original nickname documents collected after manual review, formatted text and its corresponding processing conclusions, which are composed of training data pairs. In the entire training data set, the proportion of illegal materials is relatively small. In order to increase the training materials of positive samples, the training data is enhanced by methods such as synonym replacement and replacement based on the collection of character knowledge base. These measures help improve the model's understanding and processing capabilities of positive samples, thereby improving the overall training effect. The model relies on manual review of nickname materials for continuous training, optimization and update, and continuously improves the ability to represent text semantics.

[0056] Nickname text for review T 1 Generate text semantic vector T using text vector model 1V , format text T 2 Generate text semantic vector T using text vector model 2V .

[0057] The audit feature vector database initially stores the text semantic vector information key-value pairs generated by the text vector model for the above training materials. The key-value pairs are (K i , V i ), where key K i Represents vector features, value V i Information indicating nickname and review conclusion. Value V i Can be stored in JSON, dictionary, string, or serialized format.

[0058] For example, using string storage, the nickname is "Text1", and the vector information is K 1 , the audit conclusion is "Audit1" record, then the following key-value pair can be formed: K 1 ->"Audit1_Text1".

[0059] In a possible implementation of the present application, the text review method may also include: when the first similarity is greater than a first similarity threshold, and / or the second similarity is greater than a second similarity threshold, adding the text content, review conclusion and similarity in the review feature vector database to a vector retrieval set, the vector retrieval set being used to store the text content, review conclusion and similarity of vector features in the review feature vector database whose similarity to the text semantic vector of the text to be reviewed is greater than the first similarity threshold, and the text content, review conclusion and similarity of vector features whose similarity to the text semantic vector of the formatted text is greater than the second similarity threshold; when the review conclusions in the vector retrieval set are all normal, determining the text to be reviewed to be normal text; when the review conclusions in the vector retrieval set are all illegal, determining the text to be reviewed to be illegal text; storing the correspondence between the text to be reviewed and the review conclusion, as well as the correspondence between the formatted text and the review conclusion in the review feature vector database.

[0060] That is, if R is not empty, retrieve each value R in the result set R for the above vector i1 Judgment. If all R i1 If all R i1 If both are "violations", the text to be reviewed is determined to be in violation, and the corresponding relationship between the text to be reviewed and the review conclusion, as well as the corresponding relationship between the formatted text and the review conclusion are stored in the review feature vector database.

[0061] In a possible implementation of the present application, the text review method may also include: when the review conclusions in the vector retrieval set include normal and violation, determining that the text to be reviewed is pending text, and the review method of the text to be reviewed is personnel review; sending the text to be reviewed, formatted text, sensitive word information, text semantic vector of the text to be reviewed, and text semantic vector of the formatted text to the target personnel for review.

[0062] That is, if R is not empty, retrieve each value R in the result set R for the above vector i1 If the audit conclusions in the vector search set include both normal and illegal results, that is, all R i1Not all are "normal" and not all are "violations". The conclusion of the text to be reviewed is pending. The review method of the text to be reviewed is human review. The text to be reviewed, formatted text, sensitive word information, text semantic vector of the text to be reviewed and text semantic vector of the formatted text are sent to the target person for review. The manual review has been described in detail in the above embodiment, and this embodiment will not be repeated.

[0063] In a possible implementation of the present application, the text review method may further include: receiving the review results of the target personnel; storing the correspondence between the text to be reviewed and the review results, and the correspondence between the formatted text and the review results in the review feature vector database.

[0064] That is, after receiving the audit result of the target person, the correspondence between the text to be audited and the audit result, and the correspondence between the formatted text and the audit result can be stored in the audit feature vector database to update the audit feature vector database.

[0065] In a possible implementation of the present application, formatting the text to be reviewed to obtain the formatted text may include: inputting the text to be reviewed into a target formatting model, converting non-standard characters into standard characters, and obtaining the formatted text, wherein the target formatting model includes at least one of the following: conversion between numeric symbols and standard characters, conversion between traditional and simplified characters, conversion between variant characters and standard characters, and removal of special characters.

[0066] In other words, formatting can standardize the text in the text to be reviewed, convert the symbols in it into standard characters, convert traditional characters into simplified characters, convert deformed characters such as split characters into standard characters, and remove special characters such as spaces, slashes, backslashes, special emoticons, etc., making the formatted text easier to review.

[0067] The target formatting model may be a formatting rule engine, which has been described in detail in the above embodiments and will not be described in detail in this embodiment.

[0068] The embodiment of the present application can solve the problem of low audit accuracy of ultra-short texts. First, use the sensitive word rule engine to preliminarily determine whether the violation weight is greater than the violation weight threshold, and further perform named entity analysis on the information that exceeds the violation weight threshold. If the sensitive word is not in the nickname named entity, a secondary judgment is made in combination with the audit feature vector retrieval conclusion. If the sensitive word is in the nickname named entity, a secondary judgment is made in combination with the sensitive word disposal conclusion; for information below the violation weight threshold, a secondary judgment is made in combination with the audit feature vector retrieval conclusion. And when all the results of the audit feature vector retrieval are normal or all are violations, the audit feature vector database is updated in real time, and the nickname, nickname formatted text and its text semantic vector are added; the conclusion of the expert manual review of the nickname is updated in real time.

[0069] like Figure 2 As shown in FIG. 1 , a schematic diagram of a text review device provided in an embodiment of the present application is shown. Figure 2 As shown, the text review device may include: a formatting module 201 , a weighting module 202 and a review module 203 .

[0070] Among them, the formatting module 201 is used to format the text to be reviewed to obtain the formatted text; the weight module 202 is used to input the text to be reviewed and the formatted text into the target text review model respectively to obtain the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weight of the sensitive words in the text; the review module 203 is used to review the text to be reviewed based on the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text.

[0071] In the embodiment of the present application, first, the formatting module 201 formats the text to be reviewed to obtain the formatted text, and then the weighting module 202 inputs the text to be reviewed and the formatted text into the target text review model respectively, and obtains the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weight of the sensitive words in the text. Finally, the review module 203 reviews the text to be reviewed based on the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text. The embodiment of the present application determines sensitive words based on the text before formatting and the text after formatting, so that the determination result is more accurate.

[0072] In a possible implementation of the present application, the review module 203 is used to: respectively compare the weighted sensitive word weights of the text to be reviewed and the weighted sensitive word weights of the formatted text with the sensitive word weight threshold; based on the comparison results, determine the review method of the text to be reviewed, and the review method includes automatic review and manual review.

[0073] In one possible implementation of the present application, the review module 203 is used to: extract the named entity information of the formatted text and match the sensitive word information of the formatted text when the weighted sensitive word weight of the text to be reviewed is greater than or equal to the sensitive word weight threshold, and the weighted sensitive word weight of the formatted text is greater than or equal to the sensitive word weight threshold, wherein the named entity information includes the named entity words, the parts of speech of the named entity words, and the positions of the named entity words in the formatted text, and the sensitive word information includes the matched sensitive words, the parts of speech of the sensitive words, and the positions of the sensitive words in the formatted text; and determine the review method of the text to be reviewed based on whether the sensitive word information is included in the named entity information.

[0074] In one possible implementation of the present application, the audit module 203 is used to: when all sensitive words in the sensitive word information are included in the named entity information, determine that the audit method of the text to be audited is automatic audit, and determine that the text to be audited is an illegal text; store the correspondence between the text to be audited and the audit conclusion, and the correspondence between the formatted text and the audit conclusion in an audit feature vector database, and the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

[0075] In a possible implementation of the present application, the review module 203 is used to: when the sensitive words in the sensitive word information are not included or not all included in the named entity information, determine that the review method for the text to be reviewed is human review; and send the text to be reviewed, the formatted text and the sensitive word information to the target person for review.

[0076] In a possible implementation of the present application, the audit module 203 is used to: when the weighted sensitive word weight of the text to be audited is less than the sensitive word weight threshold, and / or the weighted sensitive word weight of the formatted text is less than the sensitive word weight threshold, input the text to be audited and the formatted text into the target text vector model to obtain the text semantic vector of the text to be audited and the text semantic vector of the formatted text; determine the first similarity between the text semantic vector of the text to be audited and each vector feature in the audit feature vector database, and the second similarity between the text semantic vector of the formatted text and each vector feature in the audit feature vector database; when the first similarity is less than or equal to the first similarity threshold, and the second similarity is less than or equal to the second similarity threshold, determine that the audit method of the text to be audited is automatic audit, and determine that the text to be audited is normal text; store the correspondence between the text to be audited and the audit conclusion, and the correspondence between the formatted text and the audit conclusion in the audit feature vector database, and the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

[0077] In a possible implementation of the present application, the audit module 203 is used to: when the first similarity is greater than a first similarity threshold, and / or the second similarity is greater than a second similarity threshold, add the text content, audit conclusion and similarity in the audit feature vector database to the vector retrieval set, the vector retrieval set being used to store the text content, audit conclusion and similarity of the vector features in the audit feature vector database whose similarity with the text semantic vector of the text to be audited is greater than the first similarity threshold, and the text content, audit conclusion and similarity of the vector features whose similarity with the text semantic vector of the formatted text is greater than the second similarity threshold; when the audit conclusions in the vector retrieval set are all normal, determine that the text to be audited is normal text; when the audit conclusions in the vector retrieval set are all illegal, determine that the text to be audited is illegal text; store the correspondence between the text to be audited and the audit conclusion, and the correspondence between the formatted text and the audit conclusion in the audit feature vector database.

[0078] In a possible implementation of the present application, the audit module 203 is used to: when the audit conclusions in the vector retrieval set include normal and illegal, determine that the text to be audited is pending text, and the audit method of the text to be audited is personnel review; send the text to be audited, formatted text, sensitive word information, text semantic vector of the text to be audited, and text semantic vector of the formatted text to the target person for review.

[0079] In a possible implementation of the present application, the audit module 203 is used to: receive the audit result of the target person; and store the correspondence between the text to be audited and the audit result, and the correspondence between the formatted text and the audit result in the audit feature vector database.

[0080] In a possible implementation of the present application, the formatting module 201 is used to input the text to be reviewed into a target formatting model, convert non-standard characters into standard characters, and obtain formatted text. The target formatting model includes at least one of the following: conversion between numeric symbols and standard characters, conversion between traditional and simplified characters, conversion between variant characters and standard characters, and removal of special characters.

[0081] The function of the text review device of this application is already in Figure 1 The method embodiment shown is described in detail, so for any details not provided in the description of this embodiment, please refer to the relevant descriptions in the aforementioned embodiments, which will not be repeated here.

[0082] like Figure 3As shown, an embodiment of the present application also provides an electronic device 300, including a processor 301, a memory 302, and a program or instruction stored in the memory 302 and executable on the processor 301. When the program or instruction is executed by the processor 301, each process of the above-mentioned text review processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0083] Optionally, the embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, each process of the above-mentioned text review method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0084] Optionally, an embodiment of the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the various processes of the text review method embodiment described above are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0085] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0086] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0087] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A text review method, characterized in that: include: Format the text to be reviewed to obtain formatted text; The to-be-reviewed text and the formatted text are respectively input into the target text review model to obtain the weighted sensitive word weights of the to-be-reviewed text and the weighted sensitive word weights of the formatted text. The target text application model is used to match the sensitive words in the text and determine the weights of the sensitive words in the text; The text to be reviewed is reviewed based on the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text.

2. The method according to claim 1, characterized in that The step of reviewing the text to be reviewed based on the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text includes: Comparing the weighted sensitive word weight of the text to be reviewed and the weighted sensitive word weight of the formatted text with the sensitive word weight threshold respectively; Based on the comparison result, a review method for the text to be reviewed is determined, and the review method includes automatic review and human review.

3. The method according to claim 2, characterized in that The step of determining the review method of the text to be reviewed based on the comparison result includes: When the weighted sensitive word weight of the text to be reviewed is greater than or equal to the sensitive word weight threshold, and the weighted sensitive word weight of the formatted text is greater than or equal to the sensitive word weight threshold, extracting the named entity information of the formatted text, and matching the sensitive word information of the formatted text, the named entity information includes the named entity word, the part of speech of the named entity word, and the position of the named entity word in the formatted text, and the sensitive word information includes the matched sensitive word, the part of speech of the sensitive word, and the position of the sensitive word in the formatted text; Based on whether the sensitive word information is included in the named entity information, a review method for the text to be reviewed is determined.

4. The method according to claim 3, characterized in that The determining of the review method of the to-be-reviewed text based on whether the sensitive word information is included in the named entity information includes: In the case that all sensitive words in the sensitive word information are included in the named entity information, determining that the review method of the text to be reviewed is automatic review, and determining that the text to be reviewed is an illegal text; The correspondence between the text to be audited and the audit conclusion, and the correspondence between the formatted text and the audit conclusion are stored in an audit feature vector database, and the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

5. The method according to claim 3, characterized in that: The determining of the review method of the to-be-reviewed text based on whether the sensitive word information is included in the named entity information includes: In a case where the sensitive word in the sensitive word information is not included or not all included in the named entity information, determining that the review method of the to-be-reviewed text is human review; The text to be reviewed, the formatted text and the sensitive word information are sent to a target person for review.

6. The method according to claim 2, characterized in that The step of determining the review method of the text to be reviewed based on the comparison result includes: When the weighted sensitive word weight of the text to be reviewed is less than the sensitive word weight threshold, and / or the weighted sensitive word weight of the formatted text is less than the sensitive word weight threshold, inputting the text to be reviewed and the formatted text into a target text vector model to obtain a text semantic vector of the text to be reviewed and a text semantic vector of the formatted text; Determine a first similarity between the text semantic vector of the to-be-audited text and each vector feature in the audit feature vector database, and a second similarity between the text semantic vector of the formatted text and each vector feature in the audit feature vector database; When the first similarities are all less than or equal to a first similarity threshold, and the second similarities are all less than or equal to a second similarity threshold, determining that the review mode of the to-be-reviewed text is automatic review, and determining that the to-be-reviewed text is a normal text; The correspondence between the text to be audited and the audit conclusion, and the correspondence between the formatted text and the audit conclusion are stored in an audit feature vector database, and the audit feature vector database is used to store various illegal texts and normal texts, as well as audit conclusions.

7. The method according to claim 6, characterized in that The method further comprises: In the case that the first similarity is greater than a first similarity threshold, and / or the second similarity is greater than a second similarity threshold, the text content, the audit conclusion and the similarity in the audit feature vector database are added to a vector retrieval set, the vector retrieval set being used to store the text content, the audit conclusion and the similarity of the vector features in the audit feature vector database whose similarity to the text semantic vector of the text to be audited is greater than the first similarity threshold, and the text content, the audit conclusion and the similarity of the vector features whose similarity to the text semantic vector of the formatted text is greater than the second similarity threshold; When the review conclusions in the vector search set are all normal, determining that the text to be reviewed is a normal text; When all the review conclusions in the vector search set are violations, determining the text to be reviewed as a violation text; The corresponding relationship between the text to be audited and the audit conclusion, and the corresponding relationship between the formatted text and the audit conclusion are stored in the audit feature vector database.

8. The method according to claim 7, characterized in that The method further comprises: When the review conclusions in the vector search set include normal and illegal, the text to be reviewed is determined to be a pending text, and the review method of the text to be reviewed is human review; The text to be reviewed, the formatted text, the sensitive word information, the text semantic vector of the text to be reviewed, and the text semantic vector of the formatted text are sent to a target person for review.

9. The method according to claim 5 or 8, characterized in that: The method further comprises: Receiving the review result of the target person; The corresponding relationship between the text to be audited and the audit result, and the corresponding relationship between the formatted text and the audit result are stored in an audit feature vector database.

10. The method according to claim 1, characterized in that The formatting of the text to be reviewed to obtain the formatted text includes: The text to be reviewed is input into a target formatting model, and non-standard characters are converted into standard characters to obtain a formatted text. The target formatting model includes at least one of the following: conversion between numeric symbols and standard characters, conversion between traditional and simplified characters, conversion between variant characters and standard characters, and removal of special characters.

Images