Method and apparatus for sensitive word variant screening based on internet community comments

Abstract

The application relates to a sensitive word screening method and device based on Internet community comments, which comprises the following steps: acquiring a total sensitive word library, and expanding the total sensitive word library to obtain an expanded sensitive word variant library; acquiring full-amount comment data in an Internet community within a preset time period; counting the first number of comment data hit by each expanded sensitive word variant in the expanded sensitive word variant library; determining the second number of target comment data hit by each expanded sensitive word variant in the expanded sensitive word variant library, which is confirmed as a violation by human review, according to the human review result of the full-amount comment data; and determining whether to take the expanded sensitive word variant as a sensitive word according to the first number and the second number corresponding to the expanded sensitive word variant. Through the technical scheme, the purpose of reducing the leakage of violation texts and reducing the violation risk of the Internet community is achieved.

Description

Technical Field

[0001] This invention relates to the field of Internet data processing technology, and in particular to a method and apparatus for filtering sensitive words based on Internet community comments. Background Technology

[0002] Due to the one-to-many nature of internet community products—meaning that content posted by one user is received by multiple users—community products often extract keywords as sensitive words based on existing violation data to identify inappropriate content in advance. To improve their risk control capabilities, community products expand their sensitive word database with numerous variations. These expanded sensitive word variations are often vast, making the ability to quickly and cost-effectively filter them crucial for reducing operational costs and enhancing risk control capabilities of internet community products.

[0003] Most internet community products rely on human review to filter extended sensitive word variations, meaning that manual review of these variations determines their suitability as sensitive words. This makes it difficult to recall high-risk data containing sensitive word variations, resulting in numerous omissions, posing risks to the platform, damaging the community atmosphere, and negatively impacting user experience and retention. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this invention provides a method and apparatus for filtering sensitive words based on internet community comments, thereby reducing the leakage of illegal texts and minimizing the risk of violations in internet communities.

[0005] According to a first aspect of the present invention, a method for filtering sensitive words based on internet community comments is provided, the method comprising: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; Obtain all comment data from online communities within a preset time period; Count the number of times each extended sensitive word variant in the aforementioned extended sensitive word variant library matches the first comment data entry; Based on the results of the human review of the full set of comment data, determine the second number of target comment data items in the extended sensitive word variant library that are confirmed to be in violation by the human review; Based on the first and second counts corresponding to the extended sensitive word variants, determine whether to designate the extended sensitive word variants as sensitive words.

[0006] In one embodiment, preferably, the total sensitive word database is expanded, including: Each sensitive word in the total sensitive word library is expanded according to a preset sensitive word expansion method, wherein the preset sensitive word expansion method includes at least one of the following: Expanding characters with similar shapes and similar sounds.

[0007] In one embodiment, preferably, counting the number of times each extended sensitive word variant in the extended sensitive word variant library matches the first comment data entry includes: Based on the full set of comment data and the extended sensitive word variant library, an extended sensitive word variant hit comment mapping table is obtained; Based on the extended sensitive word variant hit comment mapping table, calculate the first number of hit comment data for each extended sensitive word variant.

[0008] In one embodiment, preferably, determining whether to classify the extended sensitive word variant as a sensitive word based on the first and second counts corresponding to the extended sensitive word variant includes: Determine whether the number of entries corresponding to the extended sensitive word variant is greater than a first preset threshold; When the first number is greater than the first preset threshold, calculate the ratio of the second number to the first number; When the ratio is greater than the second preset threshold, the human reviewer determines whether the extended sensitive word variant is suitable as a sensitive word. When the human reviewer determines that the extended sensitive word variant is suitable as a sensitive word, the extended sensitive word variant is added to the total sensitive word library; otherwise, the extended sensitive word variant is discarded.

[0009] In one embodiment, preferably, the method further includes: Receive the first setting command input; The first preset threshold is set according to the first setting command.

[0010] In one embodiment, preferably, the method further includes: Receive the second setting command input; The second preset threshold is set according to the second setting command.

[0011] According to a second aspect of the present invention, a sensitive word filtering device based on Internet community comments is provided, the device comprising: An extension module is used to obtain a total sensitive word library and extend the total sensitive word library to obtain an extended sensitive word variant library; The acquisition module is used to acquire all comment data from internet communities within a preset time period; The statistics module is used to count the number of times each extended sensitive word variant in the extended sensitive word variant library matches the first comment data; The first determining module is used to determine, based on the human review results of the full set of comment data, the second number of target comment data entries in the extended sensitive word variant library that are confirmed to be in violation by human review for each extended sensitive word variant. The second determining module is used to determine whether to treat the extended sensitive word variant as a sensitive word based on the first number and the second number corresponding to the extended sensitive word variant.

[0012] In one embodiment, preferably, the statistical module includes: The first processing unit is used to obtain an extended sensitive word variant hit comment mapping table based on the full comment data and the extended sensitive word variant library; The first calculation unit is used to calculate the first number of each extended sensitive word variant hit comment data according to the extended sensitive word variant hit comment mapping table.

[0013] In one embodiment, preferably, the second determining module includes: The first determining unit is used to determine whether the number of the first entries corresponding to the extended sensitive word variant is greater than a first preset threshold. The second calculation unit is used to calculate the ratio of the second number to the first number when the first number is greater than the first preset threshold. The second determining unit is used to determine, when the ratio is greater than a second preset threshold, whether the extended sensitive word variant is suitable as a sensitive word; The second processing unit is used to add the extended sensitive word variant to the total sensitive word library when the human review determines that the extended sensitive word variant is suitable as a sensitive word; otherwise, it discards the extended sensitive word variant.

[0014] According to a third aspect of the present invention, a sensitive word filtering device based on Internet community comments is provided, the device comprising: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; Obtain all comment data from online communities within a preset time period; Count the number of times each extended sensitive word variant in the aforementioned extended sensitive word variant library matches the first comment data entry; Based on the results of the human review of the full set of comment data, determine the second number of target comment data items in the extended sensitive word variant library that are confirmed to be in violation by the human review; Based on the first and second counts corresponding to the extended sensitive word variants, determine whether to designate the extended sensitive word variants as sensitive words.

[0015] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in any one of the embodiments of the second aspect.

[0016] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: In this embodiment of the invention, the expanded sensitive word variant library is screened by conducting hit tests on online data, which greatly reduces the size of the expanded sensitive word variant library before it is handed over to human reviewers for verification. This can meet the need for the expanded sensitive words to be quickly and accurately added to the total sensitive word library, thereby reducing omissions and minimizing the risk of community violations.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0019] Figure 1 This is a flowchart illustrating a sensitive word filtering method based on internet community comments, according to an exemplary embodiment.

[0020] Figure 2 This is a flowchart of step S103 in a sensitive word filtering method based on Internet community comments, according to an exemplary embodiment.

[0021] Figure 3 This is a flowchart of step S105 in a sensitive word filtering method based on Internet community comments, according to an exemplary embodiment.

[0022] Figure 4 This is a block diagram illustrating a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment.

[0023] Figure 5 This is a block diagram of a statistics module in a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment.

[0024] Figure 6 This is a block diagram of a second determining module in a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0026] Figure 1 This is a flowchart illustrating a sensitive word filtering method based on internet community comments, according to an exemplary embodiment.

[0027] like Figure 1 As shown, according to a first aspect of the present invention, a method for filtering sensitive words based on internet community comments is provided, the method comprising: Step S101: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; In one embodiment, preferably, the total sensitive word database is expanded, including: Each sensitive word in the total sensitive word library is expanded according to a preset sensitive word expansion method, wherein the preset sensitive word expansion method includes at least one of the following: Expanding characters with similar shapes and similar sounds.

[0028] Step S102: Obtain all comment data from the internet community within a preset time period; For example, it is possible to obtain all online comment data within the past month.

[0029] Step S103: Count the number of times each extended sensitive word variant in the extended sensitive word variant library matches the first comment data; Among them, "hit comments" refers to the appearance of extended sensitive word variants in comments.

[0030] Step S104: Based on the review results of the full set of comment data, determine the second number of target comment data entries in the extended sensitive word variant library that are confirmed to be in violation by the reviewer for each extended sensitive word variant. Step S105: Determine whether to use the extended sensitive word variant as a sensitive word based on the first and second number of entries corresponding to the extended sensitive word variant.

[0031] In this embodiment, the expanded sensitive word variant library is screened by conducting hit tests on online data, which greatly reduces the size of the expanded sensitive word variant library before it is handed over to human review. This can meet the need for the expanded sensitive words to be quickly and accurately added to the total sensitive word library, thereby reducing omissions and minimizing the risk of community violations.

[0032] Figure 2 This is a flowchart of step S103 in a sensitive word filtering method based on Internet community comments, according to an exemplary embodiment.

[0033] like Figure 2 As shown, in one embodiment, preferably, step S103 includes: Step S201: Based on the full set of comment data and the extended sensitive word variant library, obtain the extended sensitive word variant hit comment mapping table; Step S202: Calculate the first number a of each extended sensitive word variant hit comment data according to the extended sensitive word variant hit comment mapping table.

[0034] Figure 3 This is a flowchart of step S105 in a sensitive word filtering method based on Internet community comments, according to an exemplary embodiment.

[0035] like Figure 3 As shown, in one embodiment, preferably, step S105 includes: Step S301: Determine whether the first number a corresponding to the extended sensitive word variant is greater than a first preset threshold; Step S302: When the first number a is greater than the first preset threshold, calculate the ratio b / a of the second number b to the first number a; Step S303: When the ratio is greater than the second preset threshold, the human reviewer determines whether the extended sensitive word variant is suitable as a sensitive word. Step S304: When the human reviewer determines that the extended sensitive word variant is suitable as a sensitive word, the extended sensitive word variant is added to the total sensitive word library; otherwise, the extended sensitive word variant is discarded.

[0036] In this embodiment, it is determined whether the number 'a' of comments matched by each extended sensitive word variant in the extended sensitive word variant library is greater than a threshold 'A', where 'A' needs to be determined based on the actual amount of data on the platform. Combining the number 'b' of comments confirmed as violating the rules by human reviewers for each sensitive word match with the number 'a' of comments matched by each sensitive word, b / a is calculated, and it is determined whether b / a is greater than a threshold 'B', where 'B' needs to be determined based on the false positive rate of sensitive words acceptable to human reviewers. If the number 'a' of comments matched by each extended sensitive word variant in the extended sensitive word variant library is less than or equal to threshold 'A', or b / a is less than or equal to threshold 'B', or human reviewers deem the word unsuitable as a sensitive word, then the extended sensitive word variant is discarded. Sensitive words with b / a greater than threshold 'B' are manually reviewed to confirm their suitability as sensitive words.

[0037] For example, an online simulation experiment can be conducted using real data from one month of online data to test extended sensitive word variants. If the number of times the extended sensitive word variant appears in the real online data is below a threshold A, it means that the extended sensitive word variant appears less than A times in the real online data over the past month. In other words, the extended sensitive word variant has a low probability of appearing in the real data and is therefore not suitable for inclusion in the sensitive word database. Compared to manual review, this method has the advantages of being faster and less costly.

[0038] In one embodiment, preferably, the method further includes: Receive the first setting command input; The first preset threshold is set according to the first setting command.

[0039] In this embodiment, a first preset threshold can be set based on the actual amount of comment data.

[0040] In one embodiment, preferably, the method further includes: Receive the second setting command input; The second preset threshold is set according to the second setting command.

[0041] In this embodiment, a second preset threshold can be set based on the false recall rate caused by sensitive words that are acceptable to human reviewers.

[0042] Figure 4 This is a block diagram illustrating a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment.

[0043] like Figure 4 As shown, according to a second aspect of the present invention, a sensitive word filtering device based on Internet community comments is provided, the device comprising: The extension module 41 is used to obtain the total sensitive word library and expand the total sensitive word library to obtain an extended sensitive word variant library; Module 42 is used to acquire all comment data in the Internet community within a preset time period; The statistics module 43 is used to count the number of times each extended sensitive word variant in the extended sensitive word variant library hits the first comment data; The first determining module 44 is used to determine the second number of target comment data items that are confirmed to be in violation by human review for each extended sensitive word variant in the extended sensitive word variant library, based on the human review results of the full volume of comment data; The second determining module 45 is used to determine whether to use the extended sensitive word variant as a sensitive word based on the first number and the second number corresponding to the extended sensitive word variant.

[0044] Figure 5 This is a block diagram of a statistics module in a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment.

[0045] like Figure 5 As shown, in one embodiment, preferably, the statistics module 43 includes: The first processing unit 51 is used to obtain an extended sensitive word variant hit comment mapping table based on the full comment data and the extended sensitive word variant library; The first calculation unit 52 is used to calculate the first number of each extended sensitive word variant hit comment data according to the extended sensitive word variant hit comment mapping table.

[0046] Figure 6 This is a block diagram of a second determining module in a sensitive word filtering device based on Internet community comments, according to an exemplary embodiment.

[0047] like Figure 6 As shown, in one embodiment, preferably, the second determining module includes: The first determining unit 61 is used to determine whether the number of the first entries corresponding to the extended sensitive word variant is greater than a first preset threshold. The second calculation unit 62 is used to calculate the ratio of the second number to the first number when the first number is greater than the first preset threshold. The second determining unit 63 is used to determine whether the extended sensitive word variant is suitable as a sensitive word when the ratio is greater than the second preset threshold. The second processing unit 64 is used to add the extended sensitive word variant to the total sensitive word library when the human review determines that the extended sensitive word variant is suitable as a sensitive word; otherwise, the extended sensitive word variant is discarded.

[0048] According to a third aspect of the present invention, a sensitive word filtering device based on Internet community comments is provided, the device comprising: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; Obtain all comment data from online communities within a preset time period; Count the number of times each extended sensitive word variant in the aforementioned extended sensitive word variant library matches the first comment data entry; Based on the results of the human review of the full set of comment data, determine the second number of target comment data items in the extended sensitive word variant library that are confirmed to be in violation by the human review; Based on the first and second counts corresponding to the extended sensitive word variants, determine whether to designate the extended sensitive word variants as sensitive words.

[0049] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method as described in any one of the embodiments of the second aspect.

[0050] Furthermore, it can be understood that in this invention, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0051] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this invention, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0052] It is further understood that although the operations are described in a specific order in the accompanying drawings in the embodiments of the present invention, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all the operations shown to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0053] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0054] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for filtering sensitive words based on internet community comments, characterized in that, The method includes: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; Obtain all comment data from online communities within a preset time period; Count the number of times each extended sensitive word variant in the aforementioned extended sensitive word variant library matches the first comment data entry; Based on the results of the human review of the full set of comment data, determine the second number of target comment data items in the extended sensitive word variant library that are confirmed to be in violation by the human review; Based on the first and second counts corresponding to the extended sensitive word variants, determine whether to designate the extended sensitive word variants as sensitive words; The expansion of the total sensitive word database includes: Each sensitive word in the total sensitive word library is expanded according to a preset sensitive word expansion method, wherein the preset sensitive word expansion method includes at least one of the following: Expanding similar-looking characters and expanding similar-sounding characters: The number of times each extended sensitive word variant in the aforementioned library matches the first comment is counted, including: Based on the full set of comment data and the extended sensitive word variant library, an extended sensitive word variant hit comment mapping table is obtained; Based on the extended sensitive word variant hit comment mapping table, calculate the first number of each extended sensitive word variant hit comment data; The step of determining whether to classify the extended sensitive word variant as a sensitive word based on the first and second counts corresponding to the extended sensitive word variant includes: Determine whether the number of entries corresponding to the extended sensitive word variant is greater than a first preset threshold; When the first number is greater than the first preset threshold, calculate the ratio of the second number to the first number; When the ratio is greater than the second preset threshold, the human reviewer determines whether the extended sensitive word variant is suitable as a sensitive word. When the human reviewer determines that the extended sensitive word variant is suitable as a sensitive word, the extended sensitive word variant is added to the total sensitive word library; otherwise, the extended sensitive word variant is discarded.

2. The method according to claim 1, characterized in that, The method further includes: Receive the first setting command input; The first preset threshold is set according to the first setting command.

3. The method according to claim 1, characterized in that, The method further includes: Receive the second setting command input; The second preset threshold is set according to the second setting command.

4. A sensitive word filtering device based on internet community comments, characterized in that, The device includes: An extension module is used to obtain a total sensitive word library and extend the total sensitive word library to obtain an extended sensitive word variant library; The acquisition module is used to acquire all comment data from internet communities within a preset time period; The statistics module is used to count the number of times each extended sensitive word variant in the extended sensitive word variant library matches the first comment data; The first determining module is used to determine, based on the human review results of the full set of comment data, the second number of target comment data entries in the extended sensitive word variant library that are confirmed to be in violation by human review for each extended sensitive word variant. The second determining module is used to determine whether to use the extended sensitive word variant as a sensitive word based on the first number and the second number corresponding to the extended sensitive word variant; The statistics module includes: The first processing unit is used to obtain an extended sensitive word variant hit comment mapping table based on the full comment data and the extended sensitive word variant library; The first calculation unit is used to calculate the first number of each extended sensitive word variant hit comment data according to the extended sensitive word variant hit comment mapping table; The second determining module includes: The first determining unit is used to determine whether the number of the first entries corresponding to the extended sensitive word variant is greater than a first preset threshold. The second calculation unit is used to calculate the ratio of the second number to the first number when the first number is greater than the first preset threshold. The second determining unit is used to determine, when the ratio is greater than a second preset threshold, whether the extended sensitive word variant is suitable as a sensitive word; The second processing unit is used to add the extended sensitive word variant to the total sensitive word library when the human review determines that the extended sensitive word variant is suitable as a sensitive word; otherwise, it discards the extended sensitive word variant.

5. A sensitive word filtering device based on internet community comments, characterized in that, The device includes: processor; Memory used to store processor-executable instructions; The processor is configured as follows: Obtain the total sensitive word library and expand the total sensitive word library to obtain an expanded sensitive word variant library; Obtain all comment data from online communities within a preset time period; Count the number of times each extended sensitive word variant in the aforementioned extended sensitive word variant library matches the first comment data entry; Based on the results of the human review of the full set of comment data, determine the second number of target comment data items in the extended sensitive word variant library that are confirmed to be in violation by the human review; Based on the first and second counts corresponding to the extended sensitive word variants, determine whether to designate the extended sensitive word variants as sensitive words; The expansion of the total sensitive word database includes: Each sensitive word in the total sensitive word library is expanded according to a preset sensitive word expansion method, wherein the preset sensitive word expansion method includes at least one of the following: Expanding similar-looking characters and expanding similar-sounding characters: The number of times each extended sensitive word variant in the aforementioned library matches the first comment is counted, including: Based on the full set of comment data and the extended sensitive word variant library, an extended sensitive word variant hit comment mapping table is obtained; Based on the extended sensitive word variant hit comment mapping table, calculate the first number of each extended sensitive word variant hit comment data; The step of determining whether to classify the extended sensitive word variant as a sensitive word based on the first and second counts corresponding to the extended sensitive word variant includes: Determine whether the number of entries corresponding to the extended sensitive word variant is greater than a first preset threshold; When the first number is greater than the first preset threshold, calculate the ratio of the second number to the first number; When the ratio is greater than the second preset threshold, the human reviewer determines whether the extended sensitive word variant is suitable as a sensitive word. When the human reviewer determines that the extended sensitive word variant is suitable as a sensitive word, the extended sensitive word variant is added to the total sensitive word library; otherwise, the extended sensitive word variant is discarded.

Images