Text processing method and apparatus, and electronic device

Abstract

This disclosure provides a text processing method involving technical fields such as content moderation and sensitive word matching. The specific solution involves: obtaining the text to be published and a reference word list including multiple sensitive words and their corresponding first weights; based on the reference word list, traversing the text to be published to determine the set of target sensitive words contained within it; if the number of target sensitive words in the target sensitive word set is less than a first threshold, and the number of header target sensitive words is less than a second threshold, processing the text to be published using a text processing model to determine its sensitivity probability; and determining whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word. This improves the reliability of text processing.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of content risk control and sensitive word detection, and specifically to text processing methods, devices and electronic devices. Background Technology

[0002] With the advent of the internet age, the vast amount of online resources has made people's daily lives, social interactions, studies, and work increasingly convenient and efficient. However, while enjoying the convenience brought by the internet, many people also use it to publish harmful information, causing numerous negative impacts. Therefore, reviewing and filtering published content is crucial. Summary of the Invention

[0003] This disclosure provides a text processing method, apparatus, and electronic device.

[0004] According to one aspect of this disclosure, a text processing method is provided, comprising:

[0005] Obtain the text to be published and a reference thesaurus, where the reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word;

[0006] Based on the reference vocabulary, the text to be published is traversed to determine the set of target sensitive words contained in the text to be published.

[0007] When the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the text processing model is used to process the text to be published and determine the sensitivity probability of the text to be published. Among them, the header target sensitive words are target sensitive words with a first weight greater than the third threshold.

[0008] Based on the first weight and sensitivity probability of each target sensitive word, determine whether to block the text to be published.

[0009] According to another aspect of this disclosure, a text processing apparatus is provided, comprising:

[0010] The acquisition module is used to acquire the text to be published and a reference thesaurus. The reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word.

[0011] The determination module is used to traverse the text to be published based on a reference thesaurus to determine the set of target sensitive words contained in the text to be published.

[0012] The prediction module is used to process the text to be published using a text processing model when the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, thereby determining the sensitivity probability of the text to be published. Here, header target sensitive words are target sensitive words with a first weight greater than a third threshold.

[0013] The processing module is used to determine whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word.

[0014] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described in the above embodiments.

[0018] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in the above embodiments.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 A flowchart illustrating a text processing method provided in an embodiment of this disclosure;

[0022] Figure 2 A flowchart illustrating another text processing method provided in this disclosure embodiment;

[0023] Figure 3 A flowchart illustrating another text processing method provided in this disclosure embodiment;

[0024] Figure 4 A flowchart illustrating another text processing method provided in this disclosure embodiment;

[0025] Figure 5 This is a schematic diagram of the structure of another text processing apparatus provided in an embodiment of the present disclosure;

[0026] Figure 6 This is a block diagram of an electronic device used to implement text processing according to embodiments of the present disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] Artificial intelligence (AI) is the study of using computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies include computer vision, speech recognition, natural language processing, deep learning, big data processing, and knowledge graph technologies.

[0029] In this disclosure, when the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, it is determined whether to block the text to be published based on the first weight of each target sensitive word and the sensitivity probability determined based on the overall semantics of the text to be published, thereby improving the reliability of text processing.

[0030] The text processing methods, apparatus, electronic devices, and storage media of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0031] It should be noted that the text processing method of this disclosure is illustrated by being configured in a text processing device (hereinafter referred to as the processing device). The processing device can be applied to any electronic device so that the electronic device can perform the function of reviewing the text to be published.

[0032] Among them, electronic devices can be any device with computing capabilities, such as personal computers (PCs), mobile terminals, etc. Mobile terminals can be hardware devices with various operating systems, touch screens and / or displays, such as mobile phones, tablets, personal digital assistants, wearable devices, etc.

[0033] Figure 1 This is a flowchart illustrating a text processing method provided in an embodiment of the present disclosure.

[0034] like Figure 1 As shown, the method includes:

[0035] Step 101: Obtain the text to be published and a reference thesaurus. The reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word.

[0036] The first weight can be used to indicate the importance of a sensitive word. The first weight of a sensitive word can be determined based on the probability of its appearance. The higher the first weight of a sensitive word, the greater the likelihood that the text containing that word is sensitive text.

[0037] Furthermore, the primary weight of sensitive words may differ across different types of text blocking and censorship. For example, "gambling" has a higher primary weight in gambling-related texts, while it has a lower primary weight in abusive texts, or even not at all in abusive texts. Therefore, different reference word lists can be generated based on different types of text blocking and censorship to improve the reliability of text blocking.

[0038] In this disclosure, after a user submits text to be published through a client, the processing device can obtain the text to be published and, according to the pre-set text blocking and review type, obtain the reference vocabulary corresponding to that blocking and review type.

[0039] Step 102: Based on the reference vocabulary, traverse the text to be published to determine the set of target sensitive words contained in the text to be published.

[0040] In this disclosure, any pre-set sensitive word matching algorithm, such as the AC (Aho-Corasickautomaton) algorithm, can be used to traverse the text to be published based on a reference vocabulary to determine the set of target sensitive words contained in the text to be published.

[0041] Step 103: When the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text to be published. Here, the header target sensitive words are target sensitive words with a first weight greater than the third threshold.

[0042] In this disclosure, if the number of target sensitive words exceeds a first threshold, or the number of header target sensitive words exceeds a second threshold, it indicates that the text to be published is sensitive text. In this case, the text to be published can be directly blocked. This improves the efficiency of text processing.

[0043] If the number of target sensitive words in the target sensitive word set is less than the first threshold, and the number of header target sensitive words is less than the second threshold, in order to improve the reliability of text processing, a text processing model can be used to further process the text to be published based on the overall semantics of the text, and determine the sensitivity probability of the text to be published. Then, based on the sensitivity probability of the text to be published, it is determined whether to block the text.

[0044] Step 104: Determine whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word.

[0045] In this disclosure, the sensitivity probability is the likelihood that the text to be published is sensitive text, determined based on the semantics of the text to be published. The first weight characterizes the sensitivity of a single word, and the first weight of the target sensitive word can be manually set or adjusted according to the actual scenario, resulting in high reliability. Based on this, the sensitivity probability can be corrected according to the first weight of each target sensitive word, thereby achieving a fusion of overall and local aspects and improving the reliability of text processing.

[0046] Optionally, the ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set can be used to determine the coefficient of the first weight corresponding to each target sensitive word, and the sensitivity probability can be corrected based on the sum of the products of the first weights and coefficients of each target sensitive word.

[0047] Optionally, the sensitivity probability can be adjusted based on the maximum value in the first weight corresponding to the target sensitive word.

[0048] If the corrected probability of sensitivity is greater than a preset threshold, the text to be published is considered sensitive and will be blocked. If the corrected probability of sensitivity is less than the preset threshold, the text to be published is not considered sensitive and can be published.

[0049] In this disclosure, after obtaining the text to be published and a reference word list including multiple sensitive words and a first weight corresponding to each sensitive word, the text to be published is traversed based on the reference word list to determine the set of target sensitive words contained in the text. Then, if the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, a text processing model is used to process the text to be published to determine the sensitivity probability of the text. Then, based on the first weight and sensitivity probability of each target sensitive word, it is determined whether to block the text to be published. Thus, when the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the determination of whether to block the text to be published is based on the first weight of each target sensitive word and the sensitivity probability determined based on the overall semantics of the text to be published, thereby improving the reliability of text processing.

[0050] Figure 2 This is a flowchart illustrating a text processing method provided in an embodiment of the present disclosure.

[0051] like Figure 2 As shown, the method includes:

[0052] Step 201: Obtain the text to be published and a reference thesaurus. The reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word.

[0053] Step 202: Based on the reference vocabulary, traverse the text to be published to determine the set of target sensitive words contained in the text to be published.

[0054] Step 203: When the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text to be published. Here, the header target sensitive words are target sensitive words with a first weight greater than the third threshold.

[0055] Step 204: Determine whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word.

[0056] The specific implementation process of steps 201-204 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0057] Step 205: Obtain the update request for the first weight of the sensitive words, wherein the update request includes the sensitive words to be updated and the type of the sensitive words to be updated.

[0058] The types of sensitive words to be updated can include omitted data types, mistakenly added data types, etc. Sensitive words in mistakenly blocked texts can be set to the mistakenly added data type, while sensitive words in texts that were not blocked can be set to the omitted data type.

[0059] In this disclosure, after determining whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word, the system can save the text to be published, the set of target sensitive words for the text to be published, the sensitivity probability, and whether to block it in the system. Then, maintenance personnel can review the texts to be distributed over a period of time to identify texts that have been mistakenly blocked and those that have been missed. Next, the sensitive words in the mistakenly blocked texts and the sensitive words in the missed-blocked texts are designated as sensitive words to be updated, and the type of the sensitive words in the mistakenly blocked texts is set to the "mistakenly recruited" data type, and the type of the sensitive words in the missed-blocked texts is set to the "missed" data type. Then, through the client or human-interactive interface corresponding to the processing device, an update request can be generated and sent to the processing device based on the sensitive words to be updated and their types to update the first weight of the sensitive words in the reference word list. This improves the reliability of the first weight of the sensitive words.

[0060] Step 206: If the sensitive word to be updated is of the missing data type and the reference word list does not contain the sensitive word to be updated, add the sensitive word to be updated to the reference word list and set the first weight corresponding to the sensitive word to be updated to the default weight.

[0061] In this disclosure, to circumvent text censorship, derivative words may be used to replace sensitive words. For example, "spinach" may be used instead of "gambling." Alternatively, sensitive words may change over time and with changing social events. This could lead to some texts being missed during censorship. Maintenance personnel can review these missed texts, filter out new sensitive words, or add new sensitive words corresponding to new social events, and send update requests to add these new sensitive words to the reference word list.

[0062] Step 207: If the type of the sensitive word to be updated is a missing data type, and the reference word list contains the sensitive word to be updated, increase the first weight corresponding to the sensitive word to be updated in the reference word list.

[0063] In this disclosure, when the type of the sensitive word to be updated is an omitted data type, and the reference thesaurus contains the sensitive word to be updated, it indicates that the first weight of the sensitive word to be updated is insufficient to indicate the sensitivity of the text to be published, and the first weight of the sensitive word to be updated is set too low. In this case, the first weight corresponding to the sensitive word to be updated in the reference thesaurus can be increased by a preset value, thereby improving the reliability of the first weight.

[0064] Step 208: If the error type of the sensitive word to be updated is the mis-recruitment type, and the reference word list contains the sensitive word to be updated, reduce the first weight of the sensitive word to be updated in the reference word list.

[0065] In this disclosure, if the sensitive word to be updated is of the type of mistakenly recruited data, and the reference thesaurus contains the sensitive word to be updated, it indicates that the first weight of the sensitive word to be updated exceeds the sensitivity level of the text to be published, and the first weight of the sensitive word to be updated is set too high. In this case, the first weight corresponding to the sensitive word to be updated in the reference thesaurus can be reduced by a preset value, thereby improving the reliability of the first weight.

[0066] Optionally, the update request may also include a second weight corresponding to the sensitive word to be updated. This second weight can be used to update the first weight corresponding to the sensitive word in the reference word list. This improves the flexibility of configuring the first weight.

[0067] Optionally, if, within a preset time period, both the first weight and the updated first weight corresponding to any sensitive word are less than the sixth threshold, it indicates that the sensitive word is unimportant, and any sensitive word in the reference word list can be deleted to remove redundant data.

[0068] In this disclosure, when the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, the system determines whether to block the text to be published based on the first weight of each target sensitive word and the sensitivity probability determined based on the overall semantics of the text to be published, thereby improving the reliability of text processing.

[0069] Figure 3 This is a flowchart illustrating a text processing method provided in an embodiment of the present disclosure.

[0070] like Figure 3 As shown, the method includes:

[0071] Step 301: Obtain the text to be published and a reference thesaurus. The reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word.

[0072] Step 302: Based on the reference vocabulary, traverse the text to be published to determine the set of target sensitive words contained in the text to be published.

[0073] Step 303: When the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text to be published. Here, the header target sensitive words are target sensitive words with a first weight greater than the third threshold.

[0074] The specific implementation process of steps 301-303 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0075] Step 304: Determine the coefficient of the first weight corresponding to each target sensitive word as the ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set.

[0076] In this disclosure, the first weights corresponding to all target sensitive words in the target sensitive word set can be weighted and fused. Then, the sensitive value can be corrected based on the weighted fused value to improve the accuracy of the sensitive probability and thus improve the reliability of text processing.

[0077] Step 305: Adjust the sensitivity probability based on the sum of the products of the first weights and coefficients of each target sensitive word.

[0078] In this disclosure, the sum of the products of the first weight and coefficient of each target sensitive word can be calculated. If the sum is within a third preset range and the sensitivity probability is greater than a fifth threshold, the absolute value of the difference between the sum and the sensitivity probability can be multiplied by a preset coefficient to determine an adjustment value. Then, the sensitivity probability can be added to this adjustment value to improve the accuracy of the sensitivity probability.

[0079] Furthermore, if the sum of the products of the first weights and coefficients of each target sensitive word is within a fourth preset range, and the sensitivity probability is greater than a fifth threshold, the absolute value of the difference between this sum and the sensitivity probability can be multiplied by a preset coefficient to determine an adjustment value. Then, this adjustment value can be subtracted from the sensitivity probability to improve the accuracy of the sensitivity probability.

[0080] Among them, the minimum value of the third preset range is greater than or equal to the maximum value of the fourth preset range, and the maximum value of the third preset range is less than the third threshold.

[0081] Step 306: If the corrected sensitivity probability is greater than the fourth threshold, the text to be published is blocked.

[0082] In this disclosure, if the corrected sensitivity probability is greater than the fourth threshold, it indicates that the text to be published is sensitive and can be blocked. Conversely, if the corrected sensitivity probability is less than the fourth threshold, it indicates that the text to be published is not sensitive and can be published.

[0083] In this disclosure, after obtaining the text to be published and a reference word list including multiple sensitive words and their corresponding first weights, the text to be published is traversed based on the reference word list to determine the set of target sensitive words contained in the text. If the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, a text processing model is used to process the text to determine its sensitivity probability. Then, the ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set is determined as the coefficient of the first weight corresponding to each target sensitive word. The sensitivity probability is then corrected based on the sum of the products of the first weights and coefficients of each target sensitive word. If the corrected sensitivity probability is greater than a fourth threshold, the text to be published is blocked. Thus, when the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the sensitivity probability is corrected based on the sum of the products of the first weights and coefficients of each target sensitive word, and the decision to block the text is made based on the corrected sensitivity probability, thereby improving the reliability of text processing.

[0084] Figure 4 This is a flowchart illustrating a text processing method provided in an embodiment of the present disclosure.

[0085] like Figure 4 As shown, the method includes:

[0086] Step 401: Obtain the text to be published and a reference thesaurus. The reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word.

[0087] Step 402: Based on the reference vocabulary, traverse the text to be published to determine the set of target sensitive words contained in the text to be published.

[0088] Step 403: When the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text to be published. Here, the header target sensitive words are target sensitive words with a first weight greater than the third threshold.

[0089] The specific implementation process of steps 401-403 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0090] Step 404: If the maximum value of the first weight corresponding to the target sensitive word is within the first preset range and the sensitivity probability is greater than the fifth threshold, increase the sensitivity probability.

[0091] In this disclosure, the maximum value in the first weight corresponding to the target sensitive word can represent the highest probability that the text to be published is sensitive text. Therefore, the sensitivity probability can be adjusted based on the maximum value in the first weight corresponding to the target sensitive word, thereby reducing the complexity of the text processing method while ensuring the accuracy of text blocking.

[0092] In this disclosure, when the sensitivity of the text to be published, indicated by the first weight of the target sensitive word, is consistent with the sensitivity of the text to be published, indicated by the sensitivity probability (i.e., both the first weight and the sensitivity probability of the target sensitive word are relatively high), it indicates that the text to be published is highly likely to be sensitive. In this case, the sensitivity probability can be increased by a preset threshold to improve the reliability of text processing.

[0093] For example, if the maximum value of the first weight corresponding to the target sensitive word is between 0.6 and 0.8, the sensitivity probability is 0.75. The sensitivity probability can be increased to 0.80.

[0094] Step 405: If the maximum value of the first weight corresponding to the target sensitive word is within the second preset range and the sensitivity probability is greater than the fifth threshold, reduce the sensitivity probability.

[0095] Wherein, the minimum value of the first preset range is greater than or equal to the maximum value of the second preset range, and the maximum value of the first preset range is less than the third threshold.

[0096] In this disclosure, when the first weight of the target sensitive word is very small, the probability that the text to be published is also relatively small. Conversely, when the first weight of the target sensitive word is small and the sensitivity probability is large, it indicates that the sensitivity probability is too high. In this case, the sensitivity probability can be reduced by a preset threshold to improve the reliability of text processing.

[0097] For example, if the maximum value of the first weight corresponding to the target sensitive word is below 0.3, the sensitivity probability is 0.80. The sensitivity probability can be reduced to 0.75.

[0098] Optionally, when the maximum value of the first weight corresponding to the target sensitive word is between the minimum value of the first preset range and the maximum value of the second preset range, the text to be published may be sensitive text or insensitive text. In this case, the sensitivity probability can be kept unchanged.

[0099] Step 406: If the corrected sensitivity probability is greater than the fourth threshold, the text to be published is blocked.

[0100] The specific implementation process of step 406 in this disclosure can be found in the detailed description of any embodiment of this disclosure, and will not be repeated here.

[0101] In this disclosure, after obtaining the text to be published and a reference word list including multiple sensitive words and their corresponding first weights, the text to be published is traversed based on the reference word list to determine the set of target sensitive words contained in the text. If the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text. Then, if the maximum value of the first weight corresponding to the target sensitive word is within a first preset range and the sensitivity probability is greater than a fifth threshold, the sensitivity probability is increased. If the maximum value of the first weight corresponding to the target sensitive word is within a second preset range and the sensitivity probability is greater than the fifth threshold, the sensitivity probability is decreased. Then, if the corrected sensitivity probability is greater than a fourth threshold, the text to be published is blocked. Thus, when the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the sensitivity probability is corrected based on the maximum value of the first weight corresponding to the target sensitive word, and the decision to block the text is made based on the corrected sensitivity probability, thereby improving the reliability of text processing.

[0102] To implement the above embodiments, this disclosure also proposes a text processing device.

[0103] Figure 5 This is a schematic diagram of the structure of a text processing device provided in an embodiment of the present disclosure.

[0104] like Figure 5 As shown, the text processing device 500 includes: an acquisition module 510, a determination module 520, a prediction module 530, and a processing module 540.

[0105] The acquisition module 510 is used to acquire the text to be published and a reference thesaurus, wherein the reference thesaurus includes multiple sensitive words and the first weight corresponding to each sensitive word;

[0106] The determination module 520 is used to traverse the text to be published based on the reference vocabulary and determine the set of target sensitive words contained in the text to be published.

[0107] The prediction module 530 is used to process the text to be published using a text processing model when the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, thereby determining the sensitivity probability of the text to be published. The header target sensitive words are target sensitive words with a first weight greater than a third threshold.

[0108] The processing module 540 is used to determine whether to block the text to be published based on the first weight and sensitivity probability of each target sensitive word.

[0109] In one possible implementation of this disclosure, an update module is further included, used for:

[0110] Get the first weight update request for sensitive words, wherein the update request includes the sensitive words to be updated and the type of the sensitive words to be updated;

[0111] If the sensitive word to be updated is of the missing data type and the reference word list does not contain the sensitive word to be updated, add the sensitive word to be updated to the reference word list and set the first weight corresponding to the sensitive word to be updated to the default weight.

[0112] If the sensitive word to be updated is a missing data type and the reference word list contains the sensitive word to be updated, increase the first weight of the sensitive word to be updated in the reference word list;

[0113] If the error type of the sensitive word to be updated is the "mistaken recruitment" type, and the reference word list contains the sensitive word to be updated, reduce the first weight of the sensitive word to be updated in the reference word list.

[0114] In one possible implementation of this disclosure, the update module is further configured to:

[0115] The update request also includes the second weight corresponding to the sensitive words to be updated. The first weight corresponding to the sensitive words to be updated in the reference word list is updated using the second weight.

[0116] In one possible implementation of this disclosure, the processing module is used for:

[0117] The ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set is used to determine the coefficient of the first weight corresponding to each target sensitive word.

[0118] The sensitivity probability is corrected based on the sum of the products of the first weight and coefficient of each target sensitive word;

[0119] If the revised sensitivity probability is greater than the fourth threshold, the text to be published will be blocked.

[0120] In one possible implementation of this disclosure, the processing module 540 includes:

[0121] If the maximum value of the first weight corresponding to the target sensitive word is within the first preset range, and the sensitivity probability is greater than the fifth threshold, then the sensitivity probability is increased.

[0122] If the maximum value of the first weight corresponding to the target sensitive word is within the second preset range and the sensitivity probability is greater than the fifth threshold, the sensitivity probability is reduced. Wherein, the minimum value of the first preset range is greater than or equal to the maximum value of the second preset range and the maximum value of the first preset range is less than the third threshold.

[0123] If the revised sensitivity probability is greater than the fourth threshold, the text to be published will be blocked.

[0124] In one possible implementation of this disclosure, the update module is further configured to:

[0125] If, within a preset time period, both the first weight and the updated first weight corresponding to any sensitive word are less than the sixth threshold, then any sensitive word in the reference word list will be deleted.

[0126] In one possible implementation of this disclosure, the processing module 540 is further configured to:

[0127] If the number of target sensitive words exceeds the first threshold, or if the number of header target sensitive words exceeds the second threshold, the text to be published will be blocked.

[0128] It should be noted that the explanation of the foregoing text processing method embodiments also applies to the apparatus of this embodiment, and therefore will not be repeated here.

[0129] In this disclosure, after obtaining the text to be published and a reference word list including multiple sensitive words and a first weight corresponding to each sensitive word, the text to be published is traversed based on the reference word list to determine the set of target sensitive words contained in the text. Then, if the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, a text processing model is used to process the text to be published to determine the sensitivity probability of the text. Then, based on the first weight and sensitivity probability of each target sensitive word, it is determined whether to block the text to be published. Thus, when the number of target sensitive words in the target sensitive word set is less than the first threshold and the number of header target sensitive words is less than the second threshold, the determination of whether to block the text to be published is based on the first weight of each target sensitive word and the sensitivity probability determined based on the overall semantics of the text to be published, thereby improving the reliability of text processing.

[0130] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.

[0131] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0132] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 602 or a computer program loaded from storage unit 608 into RAM (Random Access Memory) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. I / O (Input / Output) interface 605 is also connected to bus 604.

[0133] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0134] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as text processing methods. For example, in some embodiments, the text processing method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the text processing methods described above can be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform text processing methods by any other suitable means (e.g., by means of firmware).

[0135] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball), through which the user can...

[0139] The keyboard and pointing device are used to provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0140] The systems and techniques described herein can be implemented in computing systems that include backend components (e.g.,

[0141] A computing system that serves as a data server, or includes middleware components (e.g., an application server), or includes front-end components (e.g., a system with a graphical user interface or a web browser).

[0142] A user computer (through which the user can interact with the implementations of the systems and technologies described herein via the graphical user interface or the web browser), or a computing system including any combination of such back-end components, middleware components, or front-end components. It can be implemented in any form or medium.

[0143] Digital data communication (e.g., communication networks) is used to interconnect the components of the system. Examples of communication networks 5 include: LAN (Local Area Network), WAN (Wide Area Network), etc.

[0144] Wide area networks (WANs), the Internet, and blockchain networks.

[0145] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via a communication network. This is achieved by running on corresponding computers and having mutual...

[0146] A computer program with a client-server relationship creates the relationship between the client and the server. Server 0 can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system, which addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. A server can also be a server for a distributed system, or a server integrated with blockchain technology.

[0147] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel or sequentially.

[0148] The steps can be executed in different orders, as long as the desired result of the disclosed technical solution can be achieved, and this article does not impose any restrictions.

[0149] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A text processing method, the method comprising: Obtain the text to be published and a reference thesaurus, wherein the reference thesaurus includes multiple sensitive words and a first weight corresponding to each sensitive word; Based on the reference vocabulary, the text to be published is traversed to determine the set of target sensitive words contained in the text to be published; If the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, the text to be published is processed using a text processing model to determine the sensitivity probability of the text to be published, wherein the header target sensitive words are target sensitive words with a first weight greater than a third threshold; Based on the first weight and the sensitivity probability of each of the target sensitive words, determine whether to block the text to be published; The step of determining whether to block the text to be published based on the first weight and the sensitivity probability of each target sensitive word includes: The ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set is used to determine the coefficient of the first weight corresponding to each target sensitive word; the sensitivity probability is corrected based on the sum of the products of the first weights and coefficients of each target sensitive word; if the corrected sensitivity probability is greater than the fourth threshold, the text to be published is blocked. or, If the maximum value of the first weight corresponding to the target sensitive word is within a first preset range, and the sensitivity probability is greater than a fifth threshold, the sensitivity probability is increased; if the maximum value of the first weight corresponding to the target sensitive word is within a second preset range, and the sensitivity probability is greater than a fifth threshold, the sensitivity probability is decreased, wherein the minimum value of the first preset range is greater than or equal to the maximum value of the second preset range, and the maximum value of the first preset range is less than a third threshold; if the corrected sensitivity probability is greater than a fourth threshold, the text to be published is blocked.

2. The method as described in claim 1, wherein, Also includes: An update request for obtaining the first weight of the sensitive word is provided, wherein the update request includes the sensitive word to be updated and the type of the sensitive word to be updated; If the sensitive word to be updated is of the missing data type and the reference word list does not contain the sensitive word to be updated, add the sensitive word to be updated to the reference word list and set the first weight corresponding to the sensitive word to be updated to the default weight. If the type of the sensitive word to be updated is a missing data type, and the reference word list contains the sensitive word to be updated, increase the first weight corresponding to the sensitive word to be updated in the reference word list; If the error type of the sensitive word to be updated is the misrecruitment type, and the reference word list contains the sensitive word to be updated, then reduce the first weight corresponding to the sensitive word to be updated in the reference word list.

3. The method as described in claim 2, wherein, Also includes: The update request also includes a second weight corresponding to the sensitive word to be updated, and the first weight corresponding to the sensitive word to be updated in the reference word list is updated using the second weight.

4. The method as described in claim 1, wherein, Also includes: If, within a preset time period, both the first weight and the updated first weight corresponding to any sensitive word are less than the sixth threshold, then any sensitive word in the reference word list is deleted.

5. The method as described in claim 1, wherein, Also includes: If the number of target sensitive words exceeds a first threshold, or if the number of header target sensitive words exceeds a second threshold, the text to be published will be blocked.

6. A text processing apparatus, the apparatus comprising: The acquisition module is used to acquire the text to be published and a reference thesaurus, wherein the reference thesaurus includes multiple sensitive words and a first weight corresponding to each sensitive word; The determination module is used to traverse the text to be published based on the reference vocabulary to determine the set of target sensitive words contained in the text to be published. The prediction module is used to process the text to be published using a text processing model when the number of target sensitive words in the target sensitive word set is less than a first threshold and the number of header target sensitive words is less than a second threshold, thereby determining the sensitivity probability of the text to be published. The header target sensitive words are target sensitive words with a first weight greater than a third threshold. The processing module is used to determine whether to block the text to be published based on the first weight and the sensitivity probability of each target sensitive word; Specifically, the processing module is used to: determine the coefficient of the first weight corresponding to each target sensitive word as the ratio of the first weight corresponding to each target sensitive word to the sum of the first weights corresponding to all target sensitive words in the target sensitive word set; correct the sensitivity probability based on the sum of the products of the first weights and coefficients of each target sensitive word; and block the text to be published if the corrected sensitivity probability is greater than a fourth threshold. or, If the maximum value of the first weight corresponding to the target sensitive word is within a first preset range, and the sensitivity probability is greater than the fifth threshold, then the sensitivity probability is increased. If the maximum value of the first weight corresponding to the target sensitive word is within a second preset range, and the sensitivity probability is greater than a fifth threshold, the sensitivity probability is reduced, wherein the minimum value of the first preset range is greater than or equal to the maximum value of the second preset range, and the maximum value of the first preset range is less than the third threshold. If the corrected sensitivity probability is greater than the fourth threshold, the text to be published will be blocked.

7. The apparatus of claim 6, wherein, It also includes an update module for: An update request for obtaining the first weight of the sensitive word is provided, wherein the update request includes the sensitive word to be updated and the type of the sensitive word to be updated; If the sensitive word to be updated is of the missing data type and the reference word list does not contain the sensitive word to be updated, add the sensitive word to be updated to the reference word list and set the first weight corresponding to the sensitive word to be updated to the default weight. If the type of the sensitive word to be updated is a missing data type, and the reference word list contains the sensitive word to be updated, increase the first weight corresponding to the sensitive word to be updated in the reference word list; If the error type of the sensitive word to be updated is the misrecruitment type, and the reference word list contains the sensitive word to be updated, then reduce the first weight corresponding to the sensitive word to be updated in the reference word list.

8. The apparatus of claim 7, wherein, The update module is also used for: The update request also includes a second weight corresponding to the sensitive word to be updated, and the first weight corresponding to the sensitive word to be updated in the reference word list is updated using the second weight.

9. The apparatus of claim 7, wherein, The update module is also used for: If, within a preset time period, both the first weight and the updated first weight corresponding to any sensitive word are less than the sixth threshold, then any sensitive word in the reference word list is deleted.

10. The apparatus of claim 7, wherein, The processing module is further configured to: If the number of target sensitive words exceeds a first threshold, or if the number of header target sensitive words exceeds a second threshold, the text to be published will be blocked.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-5.

Images