Sensitive word detection method and device, electronic equipment and storage medium

Abstract

The application discloses a sensitive word detection method and device, electronic equipment and storage medium, and belongs to the field of communication, and solves the problem that the illegal words in the related art cannot be effectively intercepted. The method comprises the following steps: receiving a report message for first information sent by a first terminal corresponding to a first user; acquiring at least one reported information stored in advance and matched with a first keyword in the sensitive information from a first database set in advance according to the first keyword; if the ratio of the number of the at least one reported information to the number of all information stored in the first database is greater than a first preset threshold value, it is determined that the report is successful, and the first keyword is determined as a target sensitive word; the target sensitive word is saved to a second database set in advance to obtain an updated second database; and performing a sensitive word detection operation on a received to-be-processed message according to the updated second database.

Description

Technical Field

[0001] This application belongs to the field of communications, and specifically relates to a method, apparatus, electronic device, and storage medium for detecting sensitive words. Background Technology

[0002] In related technologies, when a server receives a message from a client, it needs to verify the presence of prohibited words in the message. If prohibited words are found, they are replaced or deleted. These prohibited words are typically pre-defined sensitive terms or terms designed to encourage consumption.

[0003] However, once the prohibited words are set in advance, they are difficult to change. The words in the messages are also rapidly evolving as information spreads online. As a result, the prohibited words in the relevant technologies are easily outdated during use, making it impossible to comprehensively intercept prohibited words in the messages. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for detecting sensitive words, which can solve the problem that illegal words cannot be effectively blocked in related technologies.

[0005] In a first aspect, embodiments of this application provide a method for detecting sensitive words. The method includes: receiving a report message sent by a first terminal corresponding to a first user regarding first information, wherein the report message includes the first information and the first information is marked with sensitive information specified by the first user; based on a first keyword in the sensitive information, retrieving at least one pre-stored reported information matching the first keyword from a pre-set first database, wherein the first database is used to store information after a successful report; if the ratio of the number of the at least one reported information to the number of all information stored in the first database is greater than a first preset threshold, then determining that the report is successful and identifying the first keyword as a target sensitive word; saving the target sensitive word to a pre-set second database to obtain an updated second database; and performing sensitive word detection on received messages to be processed based on the updated second database.

[0006] Secondly, embodiments of this application provide a sensitive word detection device, comprising: a receiving module, configured to receive a reporting message for first information sent by a first terminal corresponding to a first user, wherein the reporting message includes the first information and the first information is marked with sensitive information specified by the first user; an obtaining module, configured to obtain at least one pre-stored reported information matching the first keyword from a pre-set first database based on the first keyword in the sensitive information, wherein the first database is used to store information after a successful report; a determining module, configured to determine a successful report and identify the first keyword as a target sensitive word if the ratio of the number of the at least one reported information to the number of all information stored in the first database is greater than a first preset threshold; a saving module, configured to save the target sensitive word to a pre-set second database to obtain an updated second database; and a detection module, configured to perform sensitive word detection operations on received messages to be processed based on the updated second database.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In this embodiment, a report message for first information is received from a first terminal corresponding to a first user. The report message includes the first information, and the first information contains sensitive information specified by the first user. Based on a first keyword in the sensitive information, at least one pre-stored reported information matching the first keyword is retrieved from a pre-set first database. The first database stores information after a successful report. If the ratio of the number of the at least one reported information to the total number of all information stored in the first database is greater than a first preset threshold, the report is confirmed as successful, and the first keyword is identified as a target sensitive word. The target sensitive word is saved to a pre-set second database to obtain an updated second database. Sensitive word detection is performed on received messages based on the updated second database. This avoids the problem that sensitive words in the second database, which are pre-defined before use, are easily lost in time during use, and that illegal words (sensitive words) in the message cannot be effectively intercepted. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a sensitive word detection method provided in an embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating a sensitive word detection method provided in an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of the structure of a sensitive word detection device according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram of the structure of an electronic device according to another embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] The following description, in conjunction with the accompanying drawings, details a method, apparatus, electronic device, and storage medium for detecting sensitive words provided in this application, through specific embodiments and application scenarios.

[0018] Specifically, in related technologies, when a server receives a message from a client, it needs to verify the presence of prohibited words in the message. If prohibited words are found, they are replaced or deleted. The prohibited words in these technologies are all pre-defined sensitive words or words designed to encourage consumption.

[0019] However, once the prohibited words are set in advance, they are difficult to change. The words in the messages are also rapidly evolving as information spreads online. As a result, the prohibited words in the relevant technologies are easily outdated during use, making it impossible to comprehensively intercept prohibited words in the messages.

[0020] To address this, this application receives a report message for first information sent by a first terminal corresponding to a first user. The report message includes the first information, and the first information contains sensitive information specified by the first user. Based on a first keyword in the sensitive information, at least one pre-stored reported information matching the first keyword is retrieved from a pre-set first database. The first database stores information after a successful report. If the ratio of the number of the at least one reported information to the total number of all information stored in the first database is greater than a preset threshold, the report is confirmed as successful, and the first keyword is identified as a target sensitive word. The target sensitive word is saved to a pre-set second database, resulting in an updated second database. Sensitive word detection is performed on received messages to be processed based on the updated second database. This avoids the problem that sensitive words in the second database, which are pre-defined before use, are easily lost in time during use, thus preventing effective interception of illegal words (sensitive words) in messages.

[0021] Figure 1This illustration shows a method for detecting sensitive words according to an embodiment of the present invention. This method can be executed by an electronic device, which may include a server and / or a terminal device, wherein the terminal device may be, for example, a mobile phone terminal. In other words, the method can be executed by software or hardware installed on the electronic device, and the method includes the following steps:

[0022] Step 101: Receive the report message sent by the first terminal corresponding to the first user regarding the first information.

[0023] The report message includes the first information, and the first information is marked with sensitive information specified by the first user.

[0024] Specifically, the first piece of information can be text information or image information; no specific limitation is made here.

[0025] Optionally, the reported information is obtained by the first user reporting the first piece of information by long-pressing the report option or by a specific editor.

[0026] Optionally, sensitive information may be sensitive statements or words that the first user considers to cause discomfort, or may be advertising words that the first user considers to be advertising words; no specific limitation is made here.

[0027] Step 102: Based on the first keyword in the sensitive information, retrieve at least one reported information that matches the first keyword from a pre-set first database.

[0028] The first database is used to store information after a successful report.

[0029] Optionally, in this step, the first keyword in the sensitive information can be obtained by performing semantic analysis on the sensitive information. For example, if the sensitive information is text information, the semantics of the text information can be interpreted, and then the keywords corresponding to the text information can be interpreted based on the semantics, and these keywords can be determined as the first keyword; as another example, if the sensitive information is image information, the image information can be scanned to record image features, and then the image features can be identified to extract text information from the image, thereby allowing semantic analysis of the text information to obtain the first keyword corresponding to the sensitive information.

[0030] Specifically, the keywords corresponding to the reported information that match the first keyword can be semantically the same as or similar to the first keyword.

[0031] Step 103: If the ratio of the number of at least one reported information to the total number of information stored in the first database is greater than a first preset threshold, then the report is confirmed as successful and the first keyword is identified as a target sensitive word.

[0032] Specifically, the first preset threshold can be set by relevant technical personnel in specific applications, and is not specifically limited here.

[0033] Step 104: Save the target sensitive words to a pre-set second database to obtain the updated second database.

[0034] Optionally, the first information about a successful report can be stored in the first database.

[0035] Step 105: Perform sensitive word detection on the received message to be processed according to the updated second database.

[0036] Specifically, sensitive word detection can be performed on the received messages to be processed, that is, to detect whether there are sensitive words in the second database in the messages to be processed.

[0037] In this embodiment, by receiving a report message for first information sent by a first terminal corresponding to a first user, wherein the report message includes the first information and the first information is marked with sensitive information specified by the first user; based on the first keyword in the sensitive information, at least one reported information matching the first keyword is retrieved from a pre-set first database, wherein the first database is used to store information after a successful report; if the ratio of the number of the at least one reported information to the number of all information stored in the first database is greater than a first preset threshold, the report is determined to be successful and the first keyword is identified as a target sensitive word; the target sensitive word is saved to a pre-set second database to obtain an updated second database; and sensitive word detection is performed on the received pending messages based on the updated second database. This avoids the problem that the sensitive words in the second database are pre-set before use, which can easily lose timeliness during use, thus making it impossible to effectively intercept illegal words (sensitive words) in the message.

[0038] In one optional implementation, before receiving the report message regarding the first information sent by the first terminal corresponding to the first user, the method further includes:

[0039] The system receives second information sent by a second terminal corresponding to a second user; based on the sensitive words pre-stored in the second database, if the second information contains the sensitive words, the system filters the sensitive words in the second information to obtain first information, and sends the first information to the first terminal; if the second information does not contain sensitive information, the system determines the second information as the first information and sends it to the first terminal.

[0040] Specifically, the aforementioned sensitive words can be pre-set by relevant technical personnel.

[0041] Optionally, the technical means of filtering sensitive words in the second information can be to replace the sensitive words in the second information; optionally, the replacement words can be words that do not affect the semantic understanding of the second information, for example, the replacement words can be non-sensitive words that are semantically similar to the sensitive words.

[0042] In this way, the message received by the first user can be initially filtered for sensitive words, thus improving the user's message receiving experience.

[0043] In one optional implementation, after receiving the second information sent by the second terminal corresponding to the second user, the method further includes: determining a second keyword in the second information; determining the emotional attribute corresponding to the second keyword; if the emotional attribute is anger, determining the second keyword as a first sensitive word and the second information as first sensitive information; storing the first sensitive word in a second database and storing the first sensitive information in the first database.

[0044] Optionally, in this implementation, the second keyword in the second information can be obtained by performing semantic analysis on the second information. For example, if the second information is text information, the semantics of the text information can be interpreted, and then the keyword corresponding to the text information can be interpreted based on the semantics, and the keyword can be determined as the second keyword; as another example, if the second information is image information, the image information can be scanned to record image features, and then the image features can be identified to extract text information on the image, thereby performing semantic analysis on the text information to obtain the second keyword corresponding to the sensitive information.

[0045] Optionally, the emotional attribute of the second keyword can be analyzed based on its semantics. For example, if the second keyword is "angry", then the emotional attribute of the second keyword can be determined to be anger.

[0046] In this way, by storing the second keyword expressing anger in the second database, the sensitive words in the second database can be further updated, thereby making the sensitive words in the messages received by the user more effectively blocked.

[0047] In one optional implementation, determining the second keyword as the first sensitive word and the second information as the first sensitive information includes: receiving third information from the first user in response to the second information; determining the third keyword in the third information; determining the emotional attribute corresponding to the third keyword; if the emotional attribute is negative, determining the third information as the second sensitive information; if the ratio of the number of second sensitive information to the number of all information stored in the first database is greater than a second preset threshold, determining the second keyword as the first sensitive word and the second information as the first sensitive information.

[0048] After identifying the second keyword as the first sensitive word and the second information as the first sensitive information, the method further includes:

[0049] The second sensitive information is stored in the first database.

[0050] Optionally, in this implementation, the third keyword in the second information can be obtained by performing semantic analysis on the third information. For example, if the third information is text information, the semantics of the text information can be interpreted, and then the keyword corresponding to the text information can be interpreted based on the semantics, and the keyword can be identified as the third keyword; as another example, if the third information is image information, the image information can be scanned to record image features, and then the image features can be identified to extract text information on the image, thereby performing semantic analysis on the text information to obtain the third keyword corresponding to the sensitive information.

[0051] Optionally, the emotional attributes of the third keyword can be analyzed based on its semantics.

[0052] Optionally, negative emotions can include anger, sadness, resentment, etc., which are the opposite of positive emotions such as happiness and positivity. There is no specific limitation on this here.

[0053] Specifically, the second preset threshold can be set by relevant personnel according to actual needs in specific applications, and no specific limitations are imposed here.

[0054] In this way, by determining the second keyword as the first sensitive word when the ratio of the number of third sensitive information to the total number of information stored in the first database is greater than the second preset threshold, the sensitivity of the determined first sensitive word can be more accurate, avoiding the identification of normal words as sensitive words.

[0055] In one optional implementation, determining the emotional attribute corresponding to the third keyword includes: if the third keyword matches a pre-stored sensitive word in the second database, determining the emotional attribute corresponding to the third keyword as a negative emotion.

[0056] This makes it more accurate to determine whether the emotional attribute corresponding to the third keyword is a negative emotion, and thus makes it more accurate to determine whether the second keyword is the first sensitive word based on the emotional attribute corresponding to the third keyword.

[0057] In one alternative implementation, the sensitive words in the second database are sensitive words obtained from the Internet, wherein the sensitive words include malicious words and / or words that induce consumption.

[0058] Specifically, the second database obtains malicious words from the Internet, which can prevent users from receiving filtered messages containing such malicious words; the second database obtains consumer-promoting words from the Internet, which can prevent users from receiving filtered messages containing such consumer-promoting words.

[0059] In this way, the first database of this application can obtain sensitive words from the Internet and update the sensitive words in the first database in real time, thereby enabling the sensitive words in the message to be blocked more effectively.

[0060] Optional, see below Figure 2 The flowchart below illustrates the specific steps of one embodiment of this application.

[0061] Step 1: The second user's terminal can send instant messages to the instant messaging server. Upon arrival at the server, the server can save messages containing malicious words or promotional advertisements to the first database (X database) and can filter out such malicious words and advertisements. For example, the server can replace malicious words in the message with words that do not affect the meaning and delete advertisements in the chat window to block advertising.

[0062] Furthermore, the instant messaging server can send instant messages filtered for malicious words or advertising to the first user's terminal.

[0063] Among them, malicious words and consumer-guiding words used to identify promotional advertisements can be stored in the second database (Database A).

[0064] Step Two: Within the instant messaging server, an instant message preprocessing center can be set up, and within this center, an instant message filtering center can be configured. The filtering center can be used to replace or delete malicious keywords and promotional advertisements in instant messages. The filtering center can also include a database, database A. Database A can connect to the internet to collect statistically analyzed malicious keywords, related malicious keywords, and keywords that encourage consumption.

[0065] Specifically, before replacing or deleting malicious words and promotional advertisements in instant messages, the instant message preprocessing center can preprocess the instant messages. For example, if the instant message is an image, it can be scanned and its features recorded to identify and extract text from the image. Based on the extracted text, the meaning of the image can be interpreted, and the keywords corresponding to the extracted text can be determined. Naturally, the keywords corresponding to the image information can also be determined. If the instant message is text, its semantics can be interpreted, and the keywords corresponding to the text information can then be determined.

[0066] Naturally, based on the identified keywords for the instant message, these keywords can be compared with malicious, related, and consumer-driving words in database A to determine whether the keywords are sensitive, related, or consumer-driving words in database A. This allows us to determine whether the instant message contains sensitive or related words from database A and whether it is a promotional advertisement corresponding to consumer-driving words.

[0067] Step 3: After the first user receives the instant message on their terminal, the first user can check whether the instant message contains malicious words or promotional advertisements. If the user believes that the instant message contains malicious words or promotional advertisements, they can report the instant message.

[0068] Reporting can be done by long-pressing to select the report option, or by having a specific editor specify the submission method. Users can categorize the reported content (e.g., malicious or advertising). Specifically, users can label sensitive words that cause them discomfort or what they perceive as advertising terms as part of their report.

[0069] After a user successfully reports an instant message, the instant messaging server can read the overall semantics of the reported content and extract keywords based on the overall semantics. Furthermore, the instant messaging server can upload these keywords to the statistics center, which can check if there are identical or similar reports using the same keyword. If identical or similar reports exist, the server can determine the percentage of instant messages corresponding to that keyword among all reported messages in data X. If this percentage reaches a first preset threshold, the report is considered successful. After a successful report, malicious words and promotional advertisements on the first user message window can be replaced or deleted, and the aforementioned keywords can be stored in database A, while the reported instant message can be stored in database X.

[0070] Specifically, each sub-step in step three can be executed within the instant message reporting unit.

[0071] Step 4: If the instant messaging server determines that the instant message does not contain sensitive words and / or words that guide consumption from database A, it can immediately aggregate the instant message received by the first user, generate an aggregated text segment, and determine the keywords of the aggregated text segment. Based on this, the emotional attributes of the first user's emotions can be divided according to the semantics of the keywords of the aggregated text segment (emotional attributes may include joy, no emotional fluctuation, anger, etc.).

[0072] If the emotional attribute of the above keyword is anger, the keyword is uploaded to the statistics center. The statistics center can collect a large number of keywords corresponding to instant messages under the emotion of anger, and can extract recurring keywords, as well as keywords not stored in database A. Then, based on the negative emotion of the information replied by the first user, it can be determined that the emotional attribute of the above keyword is anger.

[0073] Specifically, the way to determine whether the sentiment of the first user's reply is negative is as follows: if the first user's reply contains sensitive words contained in database A, then the sentiment of the first user's reply can be considered negative.

[0074] If multiple instant messages received by the first user contain the aforementioned keyword, and the emotional attributes of the messages replied by the first user are all negative, and the proportion of replies with negative emotional attributes in the X database is greater than the second preset threshold, then the keyword can be identified as a sensitive word. Based on this, the keyword can be stored in the A database, and the instant messages containing the keyword and the replies to the instant messages can be stored in the X database.

[0075] Specifically, each sub-step of step four can be performed within a new vocabulary unit.

[0076] In this way, by continuously adding new sensitive words to database A in real time, sensitive words in messages sent to users can be intercepted more comprehensively, thus providing users with a better experience.

[0077] It should be noted that the sensitive word detection method provided in this application embodiment can be executed by a sensitive word detection device, or a control module in the sensitive word detection device for executing the sensitive word detection method. This application embodiment uses a sensitive word detection device executing a sensitive word detection method as an example to illustrate the sensitive word detection device provided in this application embodiment.

[0078] Figure 3 This is a schematic diagram of a sensitive word detection device according to an embodiment of the present invention. Figure 3 As shown, a sensitive word detection device 300 includes: a receiving module 310, an acquisition module 320, a determining module 330, a storage module 340, and a detection module 350.

[0079] The receiving module 310 is used to receive a report message sent by the first terminal corresponding to the first user regarding the first information, wherein the report message includes the first information and the first information is marked with sensitive information specified by the first user;

[0080] The acquisition module 320 is used to acquire at least one reported information that matches the first keyword from a pre-set first database based on the first keyword in the sensitive information, wherein the first database is used to store information after a successful report;

[0081] The determination module 330 is used to determine that the report is successful and identify the first keyword as a target sensitive word if the ratio of the number of the at least one reported information to the number of all information stored in the first database is greater than a first preset threshold.

[0082] The storage module 340 is used to save the target sensitive words to a pre-set second database to obtain an updated second database;

[0083] The detection module 350 is used to perform sensitive word detection on the received message to be processed based on the updated second database.

[0084] In one implementation, the receiving module 310 is further configured to: receive second information sent by a second terminal corresponding to a second user; based on sensitive words pre-stored in the second database, if the second information is found to contain the sensitive words, filter the sensitive words in the second information to obtain first information, and send the first information to the first terminal; if the second information is found not to contain sensitive information, determine the second information as the first information and send it to the first terminal.

[0085] In one implementation, the receiving module 310 is further configured to: determine a second keyword in the second information; determine the emotional attribute corresponding to the second keyword; if the emotional attribute is anger, determine the second keyword as a first sensitive word and the second information as first sensitive information; store the first sensitive word in a second database and store the first sensitive information in the first database.

[0086] In one implementation, the receiving module 310 is further configured to: receive third information replied by the first user in response to the second information; determine a third keyword in the third information; determine the emotional attribute corresponding to the third keyword; if the emotional attribute is a negative emotion, determine the third information as second sensitive information; if the ratio of the number of second sensitive information to the number of all information stored in the first database is greater than a second preset threshold, determine the second keyword as the first sensitive word, and determine the second information as the first sensitive information; and store the second sensitive information in the first database.

[0087] In one implementation, the receiving module 310 is further configured to: determine the emotional attribute corresponding to the third keyword as a negative emotion if the third keyword matches a pre-stored sensitive word in the second database.

[0088] In one implementation, the sensitive words in the second database are sensitive words obtained from the Internet, wherein the sensitive words include malicious words and / or words that induce consumption.

[0089] The sensitive word detection device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0090] The sensitive word detection device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0091] The sensitive word detection device provided in this application embodiment can achieve... Figures 1-2 The various processes implemented in the method embodiments are not described in detail here to avoid repetition.

[0092] Optional, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a program or instructions stored in the memory 402 and executable on the processor 401. When the program or instructions are executed by the processor 401, they implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0093] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0094] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described sensitive word detection method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0095] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0096] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described sensitive word detection method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0097] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0098] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0100] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for detecting sensitive words, characterized in that, include: Receive a report message sent by a first terminal corresponding to a first user regarding first information, wherein the report message includes the first information and the first information is marked with sensitive information specified by the first user; Based on the first keyword in the sensitive information, at least one reported information that matches the first keyword is retrieved from a pre-set first database, wherein the first database is used to store information after a successful report; If the ratio of the number of at least one reported information to the total number of information stored in the first database is greater than a first preset threshold, then the report is confirmed as successful and the first keyword is identified as a target sensitive word. The target sensitive words are saved to a pre-set second database to obtain the updated second database; The received messages to be processed are subjected to sensitive word detection based on the updated second database. Before receiving the report message regarding the first information sent by the first terminal corresponding to the first user, the method further includes: Receive the second information sent by the second terminal corresponding to the second user; Based on the sensitive words pre-stored in the second database, if the second information is detected to contain the sensitive words, the sensitive words in the second information are filtered to obtain the first information, and the first information is sent to the first terminal; If it is detected that the second information does not contain sensitive information, then the second information is identified as the first information and sent to the first terminal; After receiving the second information sent by the second terminal corresponding to the second user, the method further includes: Identify the second keyword in the second information; Determine the emotional attribute corresponding to the second keyword; When the emotional attribute is anger, the second keyword is identified as the first sensitive word, and the second information is identified as the first sensitive information; The first sensitive word is stored in the second database, and the first sensitive information is stored in the first database; The step of identifying the second keyword as the first sensitive word and the second information as the first sensitive information includes: Receive the third information in response to the second information from the first user; Determine the third keyword in the third information; Determine the emotional attribute corresponding to the third keyword; When the emotional attribute is negative, the third information is identified as the second sensitive information; If the ratio of the quantity of the second sensitive information to the quantity of all information stored in the first database is greater than the second preset threshold, the second keyword is determined as the first sensitive word, and the second information is determined as the first sensitive information.

2. The method for detecting sensitive words according to claim 1, characterized in that, After determining the second keyword as the first sensitive word and the second information as the first sensitive information, the method further includes: The second sensitive information is stored in the first database.

3. The method for detecting sensitive words according to claim 1, characterized in that, Determining the emotional attribute corresponding to the third keyword includes: If the third keyword matches a sensitive word pre-stored in the second database, the emotional attribute corresponding to the third keyword is determined to be a negative emotion.

4. The method for detecting sensitive words according to claim 1, characterized in that, The sensitive words in the second database are those obtained from the Internet, including malicious words and / or words that induce consumption.

5. A device for detecting sensitive words, characterized in that, The apparatus is used to perform the steps of the method of claim 1, including: The receiving module is used to receive a report message sent by the first terminal corresponding to the first user regarding the first information, wherein the report message includes the first information and the first information is marked with sensitive information specified by the first user; The acquisition module is used to acquire at least one reported information that matches the first keyword from a pre-set first database based on the first keyword in the sensitive information, wherein the first database is used to store information after a successful report; The determination module is used to determine that the report is successful and identify the first keyword as a target sensitive word if the ratio of the number of the at least one reported information to the total number of all information stored in the first database is greater than a first preset threshold. The saving module is used to save the target sensitive words to a pre-set second database to obtain an updated second database; The detection module is used to perform sensitive word detection on the received messages to be processed based on the updated second database.

6. The sensitive word detection device according to claim 5, characterized in that, The receiving module is also used for: Receive the second information sent by the second terminal corresponding to the second user; Based on the sensitive words pre-stored in the second database, if the second information is detected to contain the sensitive words, the sensitive words in the second information are filtered to obtain the first information, and the first information is sent to the first terminal; If it is detected that the second information does not contain sensitive information, then the second information is identified as the first information and sent to the first terminal.

7. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of the sensitive word detection method as described in any one of claims 1-4.

8. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the sensitive word detection method as described in any one of claims 1-4.

Images