Text review method, device, electronic device and storage medium

Abstract

The embodiment of the present invention relates to a text review method, device, electronic device and storage medium, including: obtaining a text to be reviewed, the text to be reviewed includes multiple paragraphs; segmenting the text to be reviewed to obtain multiple text blocks, each text block includes all or part of the text content in at least one paragraph; inputting each text block into a trained text review model, the text review model is a hierarchical multi-label classification model, obtaining a first violation label and a second violation label of each text block, the first violation label represents whether the text block contains illegal content, and the second violation label represents the violation type of the illegal content contained in the text block; based on the first violation label and the second violation label of each text block, determining the review result of each paragraph. In this way, the efficiency and accuracy of text review can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular to a text review method, device, electronic device and storage medium. Background Art

[0002] Text review is to review illegal information in texts (such as novels and papers), including but not limited to information related to politics, pornography, uncivilized language, terrorism, etc. Text review is of great significance to creating a civilized, healthy and positive reading environment.

[0003] Currently, text review is usually done manually, which results in low review efficiency and requires high labor costs. Summary of the invention

[0004] In view of this, embodiments of the present invention provide a text review method, device, electronic device and storage medium to improve text review efficiency.

[0005] In a first aspect, an embodiment of the present invention provides a text review method, comprising:

[0006] Acquire a text to be reviewed, wherein the text to be reviewed includes multiple paragraphs;

[0007] Segmenting the text to be reviewed to obtain a plurality of text blocks, each of which includes all or part of the text content in at least one paragraph;

[0008] Input each of the text blocks into a trained text audit model, which is a hierarchical multi-label classification model, to obtain a first violation label and a second violation label for each of the text blocks, wherein the first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block;

[0009] Based on the first violation label and the second violation label of each of the text blocks, the audit result of each of the paragraphs is determined, and the audit result represents whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0010] In a possible implementation manner, the segmenting of the to-be-reviewed text to obtain a plurality of text blocks includes:

[0011] For each paragraph of the text to be reviewed, determine the sum S of the paragraph length and the length M of the existing text content in the most recently created text block;

[0012] If the S does not exceed the preset text length threshold N of the text block, all text contents in the paragraph are segmented into the most recently created text block;

[0013] If the S exceeds the N, the text content of the first L bytes in the paragraph is segmented into the most recently created text block, and a new text block is created, and the remaining text content in the paragraph is segmented into the most recently created text block, and the L is the difference between the N and the M.

[0014] In a possible implementation, the text audit model includes a Bert sub-model, a global Global discrimination layer, a first local Local discrimination layer, and a second Local discrimination layer;

[0015] The step of inputting each of the text blocks into a trained text audit model to obtain a first violation label and a second violation label for each of the text blocks includes:

[0016] Each of the text blocks is input into a trained text audit model, so that the Bert sub-model performs vectorization on the text block to obtain a semantic vector of the text block and outputs it to the Global discriminant layer, so that the Global discriminant layer obtains a first hidden layer representation and a second hidden layer representation of the text block based on the semantic vector, and outputs the first hidden layer representation to the first Local discriminant layer, and outputs the second hidden layer representation to the second Local discriminant layer, so that the first Local discriminant layer obtains a first violation label based on the first hidden layer representation, and the second Local discriminant layer obtains a second violation label based on the second hidden layer representation.

[0017] In a possible implementation manner, determining the review result of each of the paragraphs based on the first violation label and the second violation label of each of the text blocks includes:

[0018] For each of the paragraphs, determining a target text block from the plurality of text blocks that contains all or part of the text content in the paragraph;

[0019] The review result of the paragraph is determined according to the first violation label and the second violation label of the target text block.

[0020] In a possible implementation manner, determining the review result of the paragraph according to the first violation label and the second violation label of the target text block includes:

[0021] If there is any target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph contains illegal content, and the violation type indicated by the audit result of the paragraph is determined according to the second violation type label corresponding to the target text block, wherein the set condition refers to: the corresponding first violation label indicates that the text block contains illegal content;

[0022] If there is no target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph does not contain any illegal content.

[0023] In a possible implementation, the method further includes: training to obtain the text review model, specifically including:

[0024] Obtaining sample data, the sample data comprising: a correspondence between a sample paragraph, a first violation label, and a second violation label;

[0025] The text review model is obtained by training with the sample data.

[0026] In a second aspect, an embodiment of the present invention provides a text review device, including:

[0027] An acquisition module, used for acquiring a text to be reviewed, wherein the text to be reviewed includes a plurality of paragraphs;

[0028] A segmentation module, used for segmenting the text to be reviewed to obtain a plurality of text blocks, each of which includes all or part of the text content in at least one paragraph;

[0029] A determination module is used to input each of the text blocks into a trained text audit model, where the text audit model is a hierarchical multi-label classification model, to obtain a first violation label and a second violation label for each of the text blocks, where the first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block;

[0030] The determination module is further used to determine the review result of each of the paragraphs based on the first violation label and the second violation label of each of the text blocks, wherein the review result represents whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the processor is used to execute a text review program stored in the memory to implement the text review method described in any one of the first aspects.

[0032] In a fourth aspect, an embodiment of the present invention provides a storage medium, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the text review method described in any one of the first aspects.

[0033] The technical solution provided by the embodiment of the present invention obtains a text to be reviewed, divides the text to be reviewed, and obtains multiple text blocks, each of which includes all or part of the text content in at least one paragraph of the text to be reviewed, and inputs each text block into a trained text review model to obtain a first violation label and a second violation label for each text block, wherein the text review model is a hierarchical multi-label classification model, the first violation label represents whether the text block contains illegal content, and the second violation label represents the violation type of the illegal content contained in the text block, and based on the first violation label and the second violation label of each text block, the review result of each paragraph is determined, the review result represents whether the paragraph contains illegal content, and when the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph, thereby realizing intelligent text review, which improves the review efficiency and reduces the review risk and cost compared with manual review. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of an embodiment of a text review method provided by an embodiment of the present invention;

[0035] Figure 2 A flowchart of an embodiment of a text review model training process provided by an embodiment of the present invention;

[0036] Figure 3 A structural diagram of a text review model provided by an embodiment of the present invention;

[0037] Figure 4 A block diagram of an embodiment of a text review device provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The text review method provided by the present invention will be explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the embodiments do not constitute a limitation of the embodiments of the present invention.

[0041] See also Figure 1 , is a flow chart of an embodiment of a text review method provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0042] Step 101: Obtain a text to be reviewed, where the text to be reviewed includes multiple paragraphs.

[0043] As an embodiment, the text audit method provided by the present invention can be deployed on a server or system platform in the form of an online service. When a user accesses the online service, he can upload the text to be audited (such as a novel, a paper, etc.), so that the server or system platform where the online service is located can obtain the audit text. This means that the execution subject of the embodiment of the present invention can be the above-mentioned server or system platform.

[0044] Step 102: segment the text to be reviewed to obtain multiple text blocks.

[0045] In the embodiment of the present invention, the text to be reviewed can be segmented according to a preset text length threshold (e.g., 256 bytes, denoted as N for convenience of description) to obtain multiple text blocks, that is, the text length of the text block does not exceed the text length threshold. It can be seen that in the embodiment of the present invention, a text block does not refer to a complete paragraph, but a text block may include all or part of the text content in at least one paragraph.

[0046] As an optional implementation method, the text to be reviewed can be segmented to obtain multiple text blocks through the following process: for each paragraph of the text to be reviewed, determine the sum (for convenience of description, denoted as S) of the paragraph length and the length of the existing text content in the most recently created text block (denoted as M for the convenience of description). Here, if it is the first paragraph of the text to be reviewed, the most recently created text block mentioned above refers to the first newly created text block, and initially, the length M of the text content in the first text block is 0; if S does not exceed the above N, it means that dividing the entire text content of the paragraph into the most recently created text block will not cause the text length of the text block to exceed N, and therefore, the entire text content of the paragraph can be segmented into the most recently created text block; if S exceeds the above N, it means that dividing the entire text content of the paragraph into the most recently created text block will cause the text length of the text block to exceed N, and therefore, part of the text content of the paragraph can be segmented into the most recently created text block, and the text length of the text block is guaranteed not to exceed N. Optionally, the first L bytes of text content in the paragraph may be segmented into the most recently created text block, where L is the difference between N and M mentioned above.

[0047] Furthermore, when S exceeds N, a new text block is created, and the remaining text content in the paragraph is segmented into the new text block.

[0048] It should be noted that, in practice, the paragraphs of the text to be reviewed are usually not too long, and the text content in a paragraph is usually segmented into two text blocks, but the situation that the paragraphs of the text to be reviewed are longer cannot be completely ruled out. In order to deal with the situation of longer paragraphs, after segmenting part of the text content in the paragraph into text blocks and creating a new text block again, it is possible to first determine whether the text length of the remaining text content in the paragraph exceeds N. If it does not exceed N, the remaining text content in the paragraph is segmented into the newly created text block. If it exceeds N, the first N bytes of the remaining text content in the paragraph are segmented into the newly created text block, and then a new text block is created. The above process is repeated until the segmentation of the paragraph is completed.

[0049] For example, assuming that the text to be reviewed includes three paragraphs, and the paragraph lengths of the three paragraphs are 256 bytes, 128 bytes, and 256 bytes respectively, and assuming that the text length threshold is 256 bytes, then, according to the segmentation method described above, the entire text content of paragraph 1 can be segmented into a text block (hereinafter referred to as text block 1), the entire text content of paragraph 2 and the first 128 bytes of the text content of paragraph 3 can be segmented into another text block (hereinafter referred to as text block 2), and the last 128 bytes of the text content of paragraph 3 can be segmented into another text block (hereinafter referred to as text block 3), and a total of three text blocks can be obtained.

[0050] For example, assuming that the text to be reviewed includes 4 paragraphs, and the paragraph lengths of the 4 paragraphs are 320 bytes, 64 bytes, 64 bytes, and 128 bytes respectively, and assuming that the text length threshold is 256 bytes, then, according to the segmentation method described above, the first 256 bytes of paragraph 1 can be segmented into a text block (hereinafter referred to as text block 4), the last 64 bytes of paragraph 1, paragraph 2, paragraph 3, and the first 64 bytes of paragraph 4 can be segmented into another text block (hereinafter referred to as text block 5), and the text content of the last 64 bytes of paragraph 4 can be segmented into another text block (hereinafter referred to as text block 6), and a total of 3 text blocks can be obtained.

[0051] Step 103: input each text block into the trained text review model to obtain a first violation label and a second violation label for each text block.

[0052] In an embodiment of the present invention, the text review model may be a hierarchical multi-label classification model for classifying whether the input text content contains illegal content, and, when the input text content contains illegal content, classifying the violation type of the illegal content.

[0053] Furthermore, when training the text review model, the sample data used may include the correspondence between the sample paragraph and the first violation label and the second violation label. The first violation label indicates whether the sample paragraph contains any violation content, and the second violation label indicates the violation type of the violation content contained in the sample paragraph. As for the specific implementation of the sample data and how to train the text review model, it will be described in the following text. Figure 2 The embodiment shown is used for illustration only and will not be described in detail here.

[0054] Based on this, in an embodiment of the present invention, a text block is input into a trained text audit model to obtain a first violation label and a second violation label of the text block. The first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block.

[0055] Step 104: Determine the review result of each paragraph based on the first violation label and the second violation label of each text block.

[0056] First of all, it is explained that the above review results represent whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0057] It can be seen from the above description that the text content in a paragraph can be divided into at least one text block, that is, for a paragraph, there may be one text block containing the entire text content of the paragraph, or there may be two or even more text blocks each containing part of the text content in the paragraph. Further, in the case where a text block contains the entire text content of the paragraph, if the text block does not contain any illegal content, it means that the paragraph does not contain any illegal content, and if the text block contains illegal content, it means that the paragraph may contain illegal content; in the case where two or even more text blocks each contain part of the text content in the paragraph, if each text block does not contain any illegal content, it means that the paragraph does not contain any illegal content, and if any text block contains illegal content, it means that the paragraph may contain illegal content. It can be seen that the audit result of a paragraph is related to the first illegal label and the second illegal label of the text block containing the text content (all or part) of the paragraph.

[0058] Based on this, in an embodiment of the present invention, for each paragraph, a text block (hereinafter referred to as a target text block) containing all or part of the text content in the paragraph is determined from the multiple text blocks obtained by segmentation, and the review result of the paragraph is determined based on the first violation label and the second violation label of the target text block.

[0059] Here, it should be noted that a text block may include all or part of the text content in multiple paragraphs of the text to be reviewed, that is, the text block where a paragraph is located is likely to include the context information of the paragraph.

[0060] For example, in the second example above, for paragraph 2 / paragraph 3, the text block in which they are located contains the context information of paragraph 2 / 3. Therefore, text review is performed at the granularity of text blocks, and then the review result of the paragraph is determined based on the review result of the text block. In this way, the review result of the paragraph can be affected by the context information of the paragraph, thereby making full use of the context information of the paragraph for text review, which can improve the accuracy of the text review results.

[0061] Specifically, as an optional implementation method, if there is any target text block that meets the set conditions, then the review result of the paragraph is determined to indicate that the paragraph contains illegal content. Here, the set conditions refer to: the corresponding first illegal label indicates that the text block contains illegal content; if there is no target text block that meets the set conditions, then the review result of the paragraph is determined to indicate that the paragraph does not contain illegal content.

[0062] For example, the first paragraph in the first example above is segmented into text block 1, that is, the target text block containing part of the text content in the paragraph is text block 1. Assuming that the first violation label of text block 1 indicates that text block 1 does not contain any illegal content, then according to the above description, it can be determined that the review result of the paragraph indicates that the paragraph does not contain any illegal content; assuming that the first violation label of text block 1 indicates that text block 1 contains illegal content, then according to the above description, it can be determined that the second review result of the paragraph indicates that the paragraph contains illegal content.

[0063] Another example: the third paragraph in the first example above is segmented into text block 2 and text block 3, that is, the target text blocks containing part of the text content in the paragraph are text block 2 and text block 3. Assuming that the first violation label of text block 2 indicates that text block 2 does not contain any illegal content, and the first violation label of text block 3 indicates that text block 2 contains illegal content, then according to the above description, it can be determined that the review result of the paragraph indicates that the paragraph contains illegal content; assuming that the first violation label of text block 2 indicates that text block 2 does not contain any illegal content, and the first violation label of text block 3 indicates that text block 2 does not contain any illegal content, then according to the above description, it can be determined that the review result of the paragraph indicates that the paragraph does not contain any illegal content.

[0064] Another example: the second paragraph in the first example above is segmented into text block 2, and text block 2 also includes part of the content of the third paragraph. It can be understood that if the first violation label of text block 2 indicates that text block 2 does not contain any violation content, it can be determined that the second paragraph does not contain any violation content either. If the first violation label of text block 2 indicates that text block 2 contains any violation content, the violation content may come from the second paragraph or the third paragraph. In this case, it is actually impossible to clearly determine whether the second paragraph contains any violation content (including or not). In this regard, according to the above description, the embodiment of the present invention adopts the following method: if any target text block corresponding to the paragraph contains any violation content, it is determined that the paragraph contains any violation content, that is, if the first violation label of text block 2 indicates that text block 2 contains any violation content, it is determined that the second paragraph contains any violation content. Through this processing, although it is possible to judge a paragraph that does not contain any violation content as containing any violation content, resulting in a misjudgment, it is possible to improve the detection rate of paragraphs containing any violation content, and avoid judging a paragraph that originally contains any violation content as not containing any violation content, resulting in major flaws in the product that quotes the paragraph.

[0065] As another optional implementation, if there is no target text block that meets the set conditions, the audit result of the paragraph is determined to indicate that the paragraph does not contain any illegal content; if there is any target text block that meets the set conditions (hereinafter referred to as the first target text block for ease of description), the number of the first target text blocks is determined, and then the audit result of the paragraph is determined based on the ratio of the first target text blocks to the total number of target text blocks. The larger the ratio, the greater the possibility that the paragraph contains illegal content, and conversely, the smaller the ratio, the smaller the possibility that the paragraph contains illegal content.

[0066] In addition, when it is determined that the audit result of a paragraph indicates that the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph can be further determined based on the second violation tag of the target text block. Specifically, as an optional implementation method, the number of occurrences of each violation type in the second violation tag of the target text block that meets the set conditions is determined, and the violation type represented by the audit result of the paragraph is determined based on the violation type with the highest number of occurrences.

[0067] The technical solution provided by the embodiment of the present invention obtains a text to be reviewed, divides the text to be reviewed, and obtains multiple text blocks, each of which includes all or part of the text content in at least one paragraph of the text to be reviewed, and inputs each text block into a trained text review model to obtain a first violation label and a second violation label for each text block, wherein the text review model is a hierarchical multi-label classification model, the first violation label represents whether the text block contains illegal content, and the second violation label represents the violation type of the illegal content contained in the text block, and based on the first violation label and the second violation label of each text block, the review result of each paragraph is determined, the review result represents whether the paragraph contains illegal content, and when the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph, thereby realizing intelligent text review, which improves the review efficiency and reduces the review risk and cost compared with manual review.

[0068] Furthermore, since a text block may include all or part of the text content in multiple paragraphs of the text to be reviewed, that is, the text block where a paragraph is located is likely to include the context information of the paragraph, in this way, text review is performed at the granularity of text blocks, and the review result of the paragraph is determined based on the review result of the text block, thereby making full use of the context information of the paragraph for text review, thereby improving the accuracy of the text review results.

[0069] See also Figure 2 , is a flow chart of an embodiment of a text audit model training process provided by an embodiment of the present invention. Figure 2 As shown, the following steps are included:

[0070] Step 201: Obtain sample data.

[0071] Step 202: Use sample data to train a text review model.

[0072] See above Figure 1 In the relevant description of the illustrated embodiment, the sample data may include: the correspondence between the sample paragraph, the first violation label, and the second violation label. For example, the sample data includes {paragraph 1, first violation label: 1 (indicating that it contains illegal content), second violation label: 16 (indicating that the violation type is pornography)}, {paragraph 2, first violation label: 1, second violation label: 32 (indicating that the violation type is terrorism)}, {paragraph 3, first violation label: 0 (indicating that it does not contain illegal content), second violation label: 0}. It should be noted that when the first violation label indicates that it does not contain illegal content, the corresponding second violation label is a fixed value, such as 0.

[0073] As an embodiment, the above sample data can be obtained manually, for example, by manually reading a novel and constructing each paragraph of each chapter in the novel into a sample data. Furthermore, after obtaining the manually constructed sample data, the sample data can be cleaned and screened by technical means to improve the accuracy of the text review model subsequently trained using the sample data. Among them, cleaning and screening include but are not limited to: cleaning special characters in the sample data, and then deleting the sample data in which the text length of the paragraph is less than a set threshold.

[0074] In addition, after obtaining the sample data, the sample data can be divided into training samples and test samples. For example, the total amount of sample data constructed is 1.8 million, of which 1.2 million are used as training samples, and the training samples are subsequently used to train the text review model; the remaining 600,000 are used as test samples, and the trained text review model is subsequently tested using the test samples, so as to fine-tune the trained text review model according to the test results and improve the accuracy of the text review model.

[0075] From the above description, it can be seen that the text audit model can be a multi-task classification model, which is used to classify whether the input text content contains illegal content, and, when the input text content contains illegal content, classify the illegal type of the illegal content. Based on this, the text audit model provided by the embodiment of the present invention can be used Figure 3 The model structure is shown in Figure 2. Figure 3As shown, the text audit model includes a Bert sub-model and an HMCN (Hierarchical Multi-Label Classification Networks) sub-model. Among them, the Bert sub-model is used to vectorize the input text content and obtain the semantic vector of the text content. The HMCN sub-model includes a Global discriminant layer and two Local discriminant layers: a first Local discriminant layer and a second Local discriminant layer. The first Local discriminant layer is used to determine whether it contains illegal content (hereinafter referred to as the illegal content discriminant layer), and the second Local discriminant layer is used to determine the illegal type of the illegal content (hereinafter referred to as the illegal type discriminant layer).

[0076] The following describes the Global discrimination layer and the two Local discrimination layers respectively:

[0077] First, the global discriminant layer corresponds to all information transmission, from the input x to each intermediate layer Finally, a fully connected layer is used to obtain the global hidden layer representation P G Here, x is the semantic vector obtained by the Bert sub-model. include

[0078] Specifically, It is used to perform hidden layer representation on whether the input x of the Global discriminant layer contains illegal content. Its input, that is, x, is usually a 768-dimensional vector, and its output is a 128-dimensional vector. The activation function used is relu. It is used to represent the violation type contained in the input x of the Global discriminant layer in the hidden layer. Its input is The result of concatenating the output vector of with x is a 128-dimensional vector, and the activation function used is relu. G It is used to represent the classification probability corresponding to the input x of the Global discriminant layer in the hidden layer. Its input is The result of concatenating the output vector of with x is a 128-dimensional vector, and the activation function used is relu.

[0079] The loss of the global discriminant layer (hereinafter referred to as loss1) is calculated by calculating the cross entropy of the classification result vector and the violation mark in the sample data as the loss value of the global discriminant layer.

[0080] Secondly, the violation discrimination layer corresponds to the local information transmission, which is transmitted from the input x to each intermediate layer Finally, a fully connected layer is used to obtain the local hidden layer representation. Here, x is The output vector of include

[0081] Specifically, Used for The output vector of contains illegal content for hidden layer representation, and its output is a 128-dimensional vector, and the activation function used is relu. Used to get The output vector of the violation result vector corresponds to the input vector of The output vector of is a 2-dimensional vector, which is the first violation label. Since there is only one classification task, softmax is used for classification.

[0082] The loss of the violation discrimination layer (hereinafter referred to as loss2) is calculated by calculating the cross entropy of the violation result vector and the violation mark in the sample data as the loss value of the violation discrimination layer.

[0083] Finally, the violation discrimination layer corresponds to the local information transmission, which is passed from the input x to each intermediate layer Finally, a fully connected layer is used to obtain the local hidden layer representation. Here, x is The output vector of include

[0084] Specifically, Used for The violation type corresponding to the output vector of is represented in the hidden layer, and its output is a 128-dimensional vector, and the activation function used is relu. Used to get The violation type vector corresponding to the output vector of The output vector is a 3-dimensional vector, which is the second violation label. Since there is only one classification task, softmax is used for classification.

[0085] The loss of the violation type discrimination layer (hereinafter referred to as loss3) is calculated by calculating the cross entropy of the violation type vector and the violation identifier in the sample data as the loss value of the violation type discrimination layer.

[0086] P F It is used to represent the classification result corresponding to the input x of the Global discriminant layer in the hidden layer. Its input is P G , The output vector is a 5-dimensional classification result vector, and sigmoid is used for multi-label classification. In the classification result vector, the first 2 dimensions are the first violation label, where it is assumed that the first violation label representing compliance is [1, 0], and the first violation label representing violation is [0, 1], and the last 3 dimensions are the second violation label. For example, the second violation label representing violation type 16 is [0, 1, 0], and the second violation label representing violation type 32 is [0, 0, 1]. The second violation label corresponding to the first violation label representing compliance is [1, 0, 0].

[0087] It should be noted that Figure 3 β in represents the weight parameter of loss1, and (1-β) represents the weight parameters of loss2 and loss3. The total loss value of the text audit model is the weighted sum of the above loss1, loss2, and loss3.

[0088] pass Figure 2 The process shown can train a multi-task classification text review model, which can not only determine whether the text content contains illegal content, but also determine the violation type of the illegal content.

[0089] Furthermore, the step 103 described above of inputting each text block into the trained text audit model to obtain the first audit result of each text block includes: inputting each text block into Figure 3 The exemplified text review model uses the Bert sub-model to vectorize the text block to obtain the semantic vector of the text block, and the Global discriminant layer obtains the first hidden layer representation of whether the text block contains illegal content and the second hidden layer representation of the violation type based on the semantic vector, and the first Local discriminant layer obtains the first violation label based on the first hidden layer representation, and the second Local discriminant layer obtains the second violation label based on the second hidden layer representation.

[0090] Corresponding to the above-mentioned embodiment of the text review method, the present invention also provides an embodiment of a text review device.

[0091] like Figure 4 As shown, it is a block diagram of an embodiment of a text review device provided by an embodiment of the present invention. The device includes: an acquisition module 41, a segmentation module 42, and a determination module 43.

[0092] The acquisition module 41 is used to acquire the text to be reviewed, and the text to be reviewed includes multiple paragraphs;

[0093] A segmentation module 42, configured to segment the text to be reviewed to obtain a plurality of text blocks, each of which includes all or part of the text content in at least one paragraph;

[0094] A determination module 43 is used to input each of the text blocks into a trained text audit model, where the text audit model is a hierarchical multi-label classification model, to obtain a first violation label and a second violation label for each of the text blocks, where the first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block;

[0095] The determination module 43 is further used to determine the review result of each of the paragraphs based on the first violation label and the second violation label of each of the text blocks, wherein the review result represents whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0096] In a possible implementation manner, the segmentation module 42 is specifically configured to:

[0097] For each paragraph of the text to be reviewed, determine the sum S of the paragraph length and the length M of the existing text content in the most recently created text block; if the S does not exceed the preset text length threshold N of the text block, then divide all the text content in the paragraph into the most recently created text block; if the S exceeds the N, then divide the text content of the first L bytes in the paragraph into the most recently created text block, and a new text block is created, and the remaining text content in the paragraph is divided into the most recently created text block, and the L is the difference between the N and the M.

[0098] In a possible implementation, the text audit model includes a Bert sub-model, a global Global discrimination layer, a first local Local discrimination layer, and a second Local discrimination layer;

[0099] The determination module 43 inputs each of the text blocks into a trained text review model to obtain a first violation label and a second violation label for each of the text blocks, including:

[0100] Each of the text blocks is input into a trained text audit model, so that the Bert sub-model performs vectorization on the text block to obtain a semantic vector of the text block and outputs it to the Global discriminant layer, so that the Global discriminant layer obtains a first hidden layer representation and a second hidden layer representation of the text block based on the semantic vector, and outputs the first hidden layer representation to the first Local discriminant layer, and outputs the second hidden layer representation to the second Local discriminant layer, so that the first Local discriminant layer obtains a first violation label based on the first hidden layer representation, and the second Local discriminant layer obtains a second violation label based on the second hidden layer representation.

[0101] In a possible implementation, the determination module 43 includes (not shown in the figure):

[0102] A target determination submodule, for determining, for each of the paragraphs, from the plurality of text blocks, a target text block containing all or part of the text content in the paragraph;

[0103] The result determination submodule is used to determine the review result of the paragraph according to the first violation label and the second violation label of the target text block.

[0104] In a possible implementation manner, the result determination submodule is specifically used to:

[0105] If there is any target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph contains illegal content, and the violation type indicated by the audit result of the paragraph is determined according to the second violation type label corresponding to the target text block, wherein the set condition refers to: the corresponding first violation label indicates that the text block contains illegal content;

[0106] If there is no target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph does not contain any illegal content.

[0107] In a possible implementation, the device further includes (not shown in the figure): a training module;

[0108] The training module is used to train the text review model, and specifically includes:

[0109] Obtaining sample data, the sample data comprising: a correspondence between a sample paragraph, a first violation label, and a second violation label;

[0110] The text review model is obtained by training with the sample data.

[0111] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, Figure 5 The electronic device 500 shown includes: at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. The various components in the electronic device 500 are coupled together via a bus system 505. It is understood that the bus system 505 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 505 is not shown in FIG. Figure 5 Various buses are labeled as bus system 505 .

[0112] The user interface 503 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0113] It can be understood that the memory 502 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0114] In some implementations, the memory 502 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 5021 and an application program 5022 .

[0115] The operating system 5021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 5022 includes various application programs, such as a media player (MediaPlayer), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 5022.

[0116] In the embodiment of the present invention, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:

[0117] Acquire a text to be reviewed, the text to be reviewed includes a plurality of paragraphs; segment the text to be reviewed to obtain a plurality of text blocks, each of the text blocks includes all or part of the text content in at least one of the paragraphs; input each of the text blocks into a trained text review model, the text review model is a hierarchical multi-label classification model, and obtain a first violation label and a second violation label for each of the text blocks, the first violation label characterizing whether the text block contains illegal content, and the second violation label characterizing the violation type of the illegal content contained in the text block; based on the first violation label and the second violation label of each of the text blocks, determine the review result of each of the paragraphs, the review result characterizing whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0118] The method disclosed in the above embodiment of the present invention can be applied to the processor 501, or implemented by the processor 501. The processor 501 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 501 or the instruction in the form of software. The above processor 501 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and completes the steps of the above method in combination with its hardware.

[0119] It is understood that the embodiments described herein can be implemented by hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application or a combination thereof.

[0120] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0121] The electronic device provided in this embodiment may be Figure 5 The electronic device shown in FIG. 1 may perform the following steps: Figure 1 , Figure 2 All steps of the Chinese text review method, thereby achieving Figure 1 , Figure 2 For details on the technical effects of the text review method shown, please refer to Figure 1 , Figure 2 For the sake of brevity, the relevant description is not repeated here.

[0122] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0123] When one or more programs in the storage medium can be executed by one or more processors, the text review method executed on the electronic device side can be implemented.

[0124] The processor is used to execute the text review program stored in the memory to implement the following steps of the text review method performed on the electronic device side:

[0125] Acquire a text to be reviewed, the text to be reviewed includes a plurality of paragraphs; segment the text to be reviewed to obtain a plurality of text blocks, each of the text blocks includes all or part of the text content in at least one of the paragraphs; input each of the text blocks into a trained text review model, the text review model is a hierarchical multi-label classification model, and obtain a first violation label and a second violation label for each of the text blocks, the first violation label characterizing whether the text block contains illegal content, and the second violation label characterizing the violation type of the illegal content contained in the text block; based on the first violation label and the second violation label of each of the text blocks, determine the review result of each of the paragraphs, the review result characterizing whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

[0126] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0127] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0128] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A text review method, characterized in that: include: Acquire a text to be reviewed, wherein the text to be reviewed includes multiple paragraphs; Segmenting the text to be reviewed to obtain a plurality of text blocks, each of which includes all or part of the text content in at least one paragraph; Input each of the text blocks into a trained text audit model, which is a hierarchical multi-label classification model, to obtain a first violation label and a second violation label for each of the text blocks, wherein the first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block; Based on the first violation label and the second violation label of each of the text blocks, the audit result of each of the paragraphs is determined, and the audit result represents whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

2. The method according to claim 1, characterized in that The text to be reviewed is segmented to obtain multiple text blocks, including: For each paragraph of the text to be reviewed, determine the sum S of the paragraph length and the length M of the existing text content in the most recently created text block; If the S does not exceed the preset text length threshold N of the text block, all text contents in the paragraph are segmented into the most recently created text block; If the S exceeds the N, the text content of the first L bytes in the paragraph is segmented into the most recently created text block, and a new text block is created, and the remaining text content in the paragraph is segmented into the most recently created text block, and the L is the difference between the N and the M.

3. The method according to claim 1, characterized in that The text audit model includes a Bert sub-model, a global Global discrimination layer, a first local Local discrimination layer, and a second Local discrimination layer; The step of inputting each of the text blocks into a trained text audit model to obtain a first violation label and a second violation label for each of the text blocks includes: Each of the text blocks is input into a trained text audit model, so that the Bert sub-model vectorizes the text block to obtain a semantic vector of the text block and outputs it to the Global discriminant layer, so that the Global discriminant layer obtains a first hidden layer representation and a second hidden layer representation of the text block based on the semantic vector, and outputs the first hidden layer representation to the first Local discriminant layer, and outputs the second hidden layer representation to the second Local discriminant layer, so that the first Local discriminant layer obtains a first violation label based on the first hidden layer representation, and the second Local discriminant layer obtains a second violation label based on the second hidden layer representation.

4. The method according to claim 1, characterized in that: The determining the review result of each of the paragraphs based on the first violation label and the second violation label of each of the text blocks includes: For each of the paragraphs, determining a target text block from the plurality of text blocks that contains all or part of the text content in the paragraph; The review result of the paragraph is determined according to the first violation label and the second violation label of the target text block.

5. The method according to claim 4, characterized in that The step of determining the review result of the paragraph according to the first violation label and the second violation label of the target text block includes: If there is any target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph contains illegal content, and the violation type indicated by the audit result of the paragraph is determined according to the second violation type label corresponding to the target text block, wherein the set condition refers to: the corresponding first violation label indicates that the text block contains illegal content; If there is no target text block that meets the set condition, it is determined that the audit result of the paragraph indicates that the paragraph does not contain any illegal content.

6. The method according to claim 1, characterized in that The method further includes: training to obtain the text review model, specifically including: Obtaining sample data, the sample data comprising: a correspondence between a sample paragraph, a first violation label, and a second violation label; The text review model is obtained by training with the sample data.

7. A text review device, characterized in that: include: An acquisition module, used for acquiring a text to be reviewed, wherein the text to be reviewed includes a plurality of paragraphs; A segmentation module, used for segmenting the text to be reviewed to obtain a plurality of text blocks, each of which includes all or part of the text content in at least one paragraph; A determination module is used to input each of the text blocks into a trained text audit model, where the text audit model is a hierarchical multi-label classification model, to obtain a first violation label and a second violation label for each of the text blocks, where the first violation label indicates whether the text block contains illegal content, and the second violation label indicates the violation type of the illegal content contained in the text block; The determination module is further used to determine the review result of each of the paragraphs based on the first violation label and the second violation label of each of the text blocks, wherein the review result represents whether the paragraph contains illegal content, and if the paragraph contains illegal content, the violation type of the illegal content contained in the paragraph.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is used to execute a text review program stored in the memory to implement the text review method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the text review method according to any one of claims 1 to 6.

Images