Automatic content screening and labeling system

Abstract

The invention discloses an automatic content screening and labeling system which comprises a content screening module used for confirming to-be-published content and performing text preprocessing and semantic analysis processing on the to-be-published content to obtain non-compliant fragments; the screening result display module is used for labeling and displaying the non-compliant fragments; when the content is published, the text content can be automatically audited, the non-compliant part can be automatically labeled, the auditing accuracy and the auditing efficiency are high, and the manual auditing cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a content automatic screening and annotation system. Background Art

[0002] At present, with the continuous development of the Internet, the number of users on various social platforms is increasing day by day. As the number of users increases, the information posted by users is also increasing, which poses a great challenge to the current review work of online platforms.

[0003] Existing audit methods mostly use manual audit, which often leads to audit omissions due to the uncontrollability of manual operations. Summary of the invention

[0004] The purpose of the present invention is to provide a content automatic screening and annotation system to solve all or one of the above problems existing in the prior art.

[0005] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: The present invention provides a content automatic screening and marking system, comprising: A content screening module is used to: confirm the content to be published, perform text preprocessing and semantic analysis on the content to be published, and obtain non-compliant fragments; The screening result display module is used to: mark and display the non-compliant fragments.

[0006] In one embodiment, the content screening module includes: a text preprocessing unit; The text preprocessing unit is used to process the content to be published by using text segmentation operations and part-of-speech tagging operations to obtain content to be identified; and extract basic review fragments based on the content to be identified.

[0007] In one of the embodiments, the text preprocessing unit is also used to extract non-compliant content in the content to be identified as a basic review fragment based on sensitive word matching and entity recognition using a deep learning model.

[0008] In one embodiment, the content screening module includes: a semantic analysis processing unit; The semantic analysis processing unit is used to perform semantic understanding processing and sentiment analysis processing on the basic audit fragment; based on the results of the semantic understanding processing and sentiment analysis processing, it is judged whether the basic audit fragment is compliant.

[0009] In one of the embodiments, the semantic analysis processing unit is further used to confirm the recognition correctness in the basic audit segment based on the semantic understanding processing and the sentiment analysis processing; and to treat the basic audit segment that does not meet the correctness as the non-compliant segment.

[0010] In one embodiment, the screening result display module includes: a highlight processing unit; The highlight processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with a highlight effect.

[0011] In one embodiment, the screening result display module includes: an underline processing unit; The underline processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with underlines.

[0012] In one embodiment, the content automatic screening and marking system further includes: a deletion processing unit; The deletion processing unit is used to determine whether the non-compliant words or phrases in the non-compliant segment are adjusted after the non-compliant words or phrases in the non-compliant segment are marked and displayed; if not, the non-compliant words or phrases in the non-compliant segment are deleted.

[0013] The beneficial effects of the technical solution of the present invention are: The content automatic screening and marking system described in the present invention can realize automatic review of text content when the content is published, and automatically mark the non-compliant parts, which has high review accuracy and review efficiency and reduces the cost of manual review. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a schematic diagram of the architecture of the automatic content screening and annotation system described in Example 1 of the present invention. DETAILED DESCRIPTION

[0016] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0017] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0018] The terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0019] This embodiment provides a content automatic screening and annotation system. Figure 1 As shown, including: A content screening module is used to: confirm the content to be published, perform text preprocessing and semantic analysis on the content to be published, and obtain non-compliant fragments; The screening result display module is used to: mark and display the non-compliant fragments.

[0020] In one embodiment, the content screening module includes: a text preprocessing unit; The text preprocessing unit is used to process the content to be published by using text segmentation operations and part-of-speech tagging operations to obtain content to be identified; and extract basic review fragments based on the content to be identified.

[0021] In one of the embodiments, the text preprocessing unit is also used to extract non-compliant content in the content to be identified as a basic review fragment based on sensitive word matching and entity recognition using a deep learning model.

[0022] In one embodiment, the content screening module includes: a semantic analysis processing unit; The semantic analysis processing unit is used to perform semantic understanding processing and sentiment analysis processing on the basic audit fragment; based on the results of the semantic understanding processing and sentiment analysis processing, it is judged whether the basic audit fragment is compliant.

[0023] In one of the embodiments, the semantic analysis processing unit is further used to confirm the recognition correctness in the basic audit segment based on the semantic understanding processing and the sentiment analysis processing; and to treat the basic audit segment that does not meet the correctness as the non-compliant segment.

[0024] In one embodiment, the screening result display module includes: a highlight processing unit; The highlight processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with a highlight effect.

[0025] In one embodiment, the screening result display module includes: an underline processing unit; The underline processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with underlines.

[0026] In one embodiment, the content automatic screening and marking system further includes: a deletion processing unit; The deletion processing unit is used to determine whether the non-compliant words or phrases in the non-compliant segment are adjusted after the non-compliant words or phrases in the non-compliant segment are marked and displayed; if not, the non-compliant words or phrases in the non-compliant segment are deleted.

[0027] Different from the existing technology, the present application adopts an automatic content screening and marking system, which can automatically review the text content when the content is published, and automatically mark the non-compliant parts. It has high review accuracy and review efficiency, and reduces the cost of manual review.

[0028] It should be understood that in the various embodiments of this document, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this document.

[0029] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0030] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0031] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0032] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

[0033] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0034] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0035] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0036] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A content automatic screening and annotation system, characterized in that: include: A content screening module is used to: confirm the content to be published, perform text preprocessing and semantic analysis on the content to be published, and obtain non-compliant fragments; The screening result display module is used to: mark and display the non-compliant fragments.

2. The content automatic screening and annotation system according to claim 1, characterized in that: The content screening module includes: a text preprocessing unit; The text preprocessing unit is used to process the content to be published by using text segmentation operations and part-of-speech tagging operations to obtain content to be identified; and extract basic review fragments based on the content to be identified.

3. The content automatic screening and annotation system according to claim 2, characterized in that: The text preprocessing unit is also used to extract non-compliant content from the content to be identified as a basic review fragment based on sensitive word matching and entity recognition using a deep learning model.

4. The automatic content screening and annotation system according to claim 1, characterized in that: The content screening module includes: a semantic analysis processing unit; The semantic analysis processing unit is used to perform semantic understanding processing and sentiment analysis processing on the basic audit fragment; based on the results of the semantic understanding processing and sentiment analysis processing, it is judged whether the basic audit fragment is compliant.

5. The automatic content screening and annotation system according to claim 4, characterized in that: The semantic analysis processing unit is also used to confirm the recognition correctness in the basic audit segment based on the semantic understanding processing and the sentiment analysis processing; and to treat the basic audit segment that does not meet the correctness as the non-compliant segment.

6. The automatic content screening and annotation system according to claim 1, characterized in that: The screening result display module includes: a highlight processing unit; The highlight processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with a highlight effect.

7. The content automatic screening and annotation system according to claim 1, characterized in that: The screening result display module includes: an underline processing unit; The underline processing unit is used to mark and display the corresponding non-compliant words or phrases in the non-compliant segment with underlines.

8. The content automatic screening and annotation system according to claim 1, characterized in that: The content automatic screening and marking system further includes: a deletion processing unit; The deletion processing unit is used to determine whether the non-compliant words or phrases in the non-compliant segment are adjusted after the non-compliant words or phrases in the non-compliant segment are marked and displayed; if not, the non-compliant words or phrases in the non-compliant segment are deleted.

Images