Content auditing method and device, electronic equipment and storage medium

Abstract

The invention provides a content auditing method and device, electronic equipment and a storage medium, and relates to the technical field of content auditing, multi-level feature extraction is performed on an original text to generate text vectors, and a self-adaptive hierarchical clustering algorithm is adopted to cluster the text vectors, so that the content auditing efficiency is improved. And meanwhile, key text features are extracted based on a clustering result to generate hierarchical violation labels, and the violation classification model is updated by utilizing new violation labels through an incremental learning mechanism. The problems that in the prior art, text variants are difficult to recognize due to insufficient keyword filtering semantic understanding, traditional hierarchical clustering calculation is high in complexity and cannot adapt to large-scale data, a machine learning classification model lacks a dynamic updating mechanism, so that new illegal content is missed, and the model is difficult to continuously adapt to new auditing requirements can be solved. The technical effects of improving the semantic recognition accuracy of the violation content, improving the real-time performance of large-scale text auditing, automatically discovering new violation categories, reducing leak detection and enhancing the expansibility and dynamic adaptability of a content auditing system are achieved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of content review, in particular to a content review method and device, an electronic device and a storage medium. BACKGROUND

[0002] Network content review is an important technical support for Internet governance, and is widely used in social media, e-commerce platforms, news information and other fields. The current user-generated content is growing exponentially, and the existing technical system mainly relies on keyword filtering, machine learning classification and traditional clustering analysis collaborative work, covering HTML tag cleaning, word segmentation processing, feature vector construction and other text preprocessing key links. Traditional hierarchical clustering algorithm cannot adapt to large-scale data processing.

[0003] The existing method has obvious shortcomings: keyword filtering lacks semantic understanding ability, and it is difficult to identify homophonic variants such as "harmonious" instead of "river crab", and it also needs frequent manual maintenance of the word library; the machine learning classification model is limited by the timeliness of the training data, and has a high risk of missing detection of new types of illegal content such as AI-generated ambiguous text; when processing millions of texts, the traditional hierarchical clustering algorithm has a quadratic growth in calculation time, resulting in a delay in review that exceeds the industry acceptable threshold. In summary, the existing technical system has systematic defects in real-time performance, semantic coverage and dynamic adaptability, and cannot meet the industry demand for rapid identification and classification of new types of illegal content. SUMMARY

[0004] The present disclosure provides a content review method and device, an electronic device and a storage medium. Its main purpose is to at least solve one of the technical problems in the related art to some extent.

[0005] According to a first aspect of the present disclosure, a content review method is provided, comprising: performing multi-level feature extraction on the original text to generate a text vector of the original text; adopting an adaptive hierarchical clustering algorithm to cluster the text vector; extracting key text features based on the clustering results, and generating a hierarchical label structure describing illegal content; obtaining the generated new illegal label, and updating the illegal classification model through an incremental learning mechanism.

[0006] Optionally, the multi-level feature extraction on the original text to generate a text vector of the original text comprises: performing character-level vector encoding on the original text to identify variants related to font and pronunciation; performing word-level vector encoding on the original text to capture context semantic features; performing phrase-level vector encoding on the original text to extract fixed collocation patterns of words; The character-level, word-level and phrase-level vector encodings are stacked to form a text vector of the original text.

[0007] Optionally, the text vector is clustered using an adaptive hierarchical clustering algorithm, including: Based on the density distribution of the text vector, the effective neighborhood range of each data point is dynamically calculated. Based on the preset geometric distance threshold and vector similarity threshold, the text vector is clustered and merged to generate a cluster.

[0008] Optionally, based on the clustering result, key text features are extracted, including: The weights of the words and phrases in the cluster are calculated, and the weights are based on the frequency of occurrence, inverse document frequency and distance from the cluster center. The highest weight of the preset number of words and phrases is selected as the key text feature.

[0009] Optionally, a hierarchical label structure describing the violation content is generated, including: The key features are input into a pre-trained language model to generate a three-level label containing violation types, violation topics and violation entities.

[0010] Optionally, the generated new violation label is obtained, and the violation classification model is updated through an incremental learning mechanism, including: Collect new violation samples and normal samples to build an incremental training set. The incremental training set is used to perform incremental learning training on the violation classification model.

[0011] Optionally, it further includes: Verify the sensitive topics in the clustering result to confirm their violation; Add the confirmed violation categories to the violation label library and update the hierarchical label structure.

[0012] According to a second aspect of the present disclosure, a content review device is provided, including: An extraction unit for performing multi-level feature extraction on the original text to generate a text vector of the original text; A clustering unit for clustering the text vector using an adaptive hierarchical clustering algorithm; A generation unit for extracting key text features based on the clustering result and generating a hierarchical label structure describing the violation content; A first update unit for obtaining the generated new violation label and updating the violation classification model through an incremental learning mechanism.

[0013] Optionally, the extraction unit is further configured to: Perform character-level vector encoding on the original text to identify shape and pronunciation-related variants; The original text is word-level vector encoded to capture contextual semantic features; The original text is phrase-level vector encoded to extract word fixed collocation patterns; The character-level, word-level and phrase-level vector encodings are superimposed to form a text vector of the original text.

[0014] Optionally, the clustering unit is further configured to: Based on the density distribution of the text vector, dynamically calculate the effective neighborhood range of each data point; Based on the preset geometric distance threshold and vector similarity threshold, the text vectors are clustered and merged to generate a cluster.

[0015] Optionally, the generation unit is further configured to: Calculate the weight of the words and phrases in the cluster, the weight is based on the frequency, inverse document frequency and distance from the cluster center; Select the highest weight of the preset number of words and phrases as the key text features.

[0016] Optionally, the generation unit is further configured to: Input the key features into the pre-trained language model to generate a three-level label containing the violation type, violation theme and violation entity.

[0017] Optionally, the update unit is further configured to: Collect new violation samples and normal samples to construct an incremental training set; Use the incremental training set to perform incremental learning training on the violation classification model.

[0018] Optionally, it further comprises: A second update unit for verifying sensitive topics in the clustering result and confirming their violation; adding the confirmed violation categories to the violation label library and updating the hierarchical label structure.

[0019] According to a third aspect of the present disclosure, an electronic device is provided, comprising: At least one processor; and The memory is in communication connection with the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the first aspect.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to make the computer execute the method of the first aspect.

[0021] According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect described above.

[0022] The content review method and device, the electronic device and the storage medium provided by the present disclosure can solve the problems in the prior art that it is difficult to identify text variants due to insufficient semantic understanding of keyword filtering, the traditional hierarchical clustering calculation has high complexity and cannot adapt to large-scale data, the machine learning classification model lacks a dynamic updating mechanism, resulting in missed detection of new violation content, and the model is difficult to continuously adapt to new review requirements, and achieve the technical effects of improving the semantic recognition accuracy of violation content, improving the real-time performance of large-scale text review, automatically discovering new violation categories and reducing missed detection, and enhancing the expansibility and dynamic adaptability of the content review system.

[0023] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them: Figure 1 A flowchart of a content review method provided by an embodiment of the present disclosure; Figure 2 A structural schematic diagram of a content review device provided by an embodiment of the present disclosure; Figure 3 A schematic block diagram of an example electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, including various details in order to facilitate understanding. They should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, descriptions of well-known functions and structures are omitted in the following description.

[0026] The content review method and device, the electronic device and the storage medium of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0027] Figure 1A flowchart of a content review method provided by an embodiment of the present disclosure is shown.

[0028] As shown in Figure 1 The method comprises the following steps: Step 101, multi-level feature extraction is performed on the original text to generate a text vector of the original text.

[0029] In an embodiment of the present disclosure, in order to comprehensively capture the semantic information of the original text and provide accurate feature support for the subsequent content review link, step 101 generates a text vector capable of representing the overall semantics of the original text by performing a multi-level feature extraction operation on the original text. Among them, multi-level feature extraction needs to start from different semantic granularity dimensions of the text, respectively extract information that can reflect the internal features of the text, each semantic granularity dimension can cover the basic unit level that can reflect the semantic of the text, and then fuse the features extracted in each dimension to finally form a text vector that can accurately reflect the semantic connotation of the original text (including possible illegal related semantic features). As an implementation, a pre-trained language model can be used to perform character-level vectorization encoding on the original text to identify homophonic variants and other deliberately evasive text variants, a bidirectional LSTM network can be used to perform word-level vector encoding on the segmented text to capture the combination relationship between words, and a convolutional neural network can be used to extract phrase-level n-gram features within a fixed window in the text. Subsequently, the vectors of the three levels of characters, words and phrases are superimposed according to the number of their occurrences in the text to obtain the text vector of the original text.

[0030] This multi-level feature extraction method can comprehensively cover the information of different semantic dimensions of the text, avoid the problem of incomplete semantic capture of the text by single-level feature extraction, and generate a text vector that can more accurately represent the semantics of the original text, thereby laying a reliable feature foundation for the subsequent accurate identification of illegal content (especially illegal content that relies on multi-dimensional semantic judgment).

[0031] Step 102, an adaptive hierarchical clustering algorithm is used to cluster the text vector.

[0032] In an embodiment of the present disclosure, to achieve efficient and accurate aggregation of large-scale text vectors and mine semantic-related text groups (to provide clustering basis for subsequent illegal content identification), step 102 performs clustering operation on the aforementioned generated text vectors using an adaptive hierarchical clustering algorithm. The adaptive hierarchical clustering algorithm needs to have the ability to dynamically adapt to data distribution, can adjust key parameters (such as data neighborhood range, cluster merging judgment condition, etc.) in the clustering process according to the actual distribution characteristics of the text vectors, at the same time, gives consideration to the geometric distance and semantic relevance of the text vectors in the clustering process, avoids the disconnection between the clustering results and the actual semantic association caused by relying on a single dimension, and needs to have the incremental processing ability to cope with new text vectors, without the need to re-execute the complete clustering process on the full amount of data to improve processing efficiency. As an implementation, the mass of text vectors can be first subjected to density analysis to automatically determine the dynamic effective neighborhood range of each data point, identify the core area with higher density in the data space and divide it into temporary micro-clusters, and at the hierarchical merging stage, determine whether to merge according to the geometric distance and semantic vector similarity between clusters, introduce approximate nearest neighbor search technology to reduce the similarity calculation complexity, and the new text vector only needs to calculate the relationship with the existing clusters to complete the clustering update.

[0033] The adaptive hierarchical clustering algorithm can solve the problems of high computational complexity, difficulty in adapting to large-scale data, and inaccurate clustering results caused by ignoring semantic relevance of the traditional hierarchical clustering algorithm, effectively improve the real-time performance and semantic accuracy of large-scale text vector clustering, and lay a foundation for accurately extracting illegal-related text groups from the clustering results.

[0034] Step 103, extract key text features based on the clustering results, and generate a hierarchical label structure describing illegal content.

[0035] In the embodiments of the present disclosure, in order to mine core information from the semantic related text groups formed by clustering, a label system that can clearly define the attributes of the illegal content is constructed, and step 103 first extracts key text features based on the clustering results, and then generates a hierarchical label structure describing the illegal content. Among them, the extraction of key text features needs to comprehensively consider the frequency of the features in the corresponding clustering group, the discrimination degree in the overall text set, and the degree of association with the core semantics of the group, and through the fusion of multi-dimensional indicators, the text features that can represent the core semantics of the clustering group (especially the potential illegal semantics) are selected; the hierarchical label structure needs to reflect the semantic level relationship from macro to micro, cover the large category division of illegal content, the specific illegal theme definition and the detail attribute description, and ensure that the label can clearly reflect the category attribution and specific characteristics of the illegal content. As an implementation, the distribution characteristics of the words and phrases in the clustering cluster can be analyzed, the weight is calculated combining the appearance frequency, inverse document frequency and cosine distance of the words / phrases and the cluster center, the top k phrases and words with higher weight and the m texts closest to the cluster center are selected, input into the model to generate a three-level label system including coarse-grained illegal type, specific illegal theme, illegal entity or expression manner, and the label explanation is attached.

[0036] This step can accurately capture the core illegal semantic of the clustering group, and the generated hierarchical label structure can clearly present the hierarchical attributes of the illegal content, which not only solves the problem that a single label is difficult to comprehensively define the illegal type, but also provides clear semantic guidance for subsequent manual review and illegal classification model updating, effectively improving the accuracy of illegal content identification and the clarity of category definition.

[0037] Step 104, obtaining the generated new illegal label, updating the illegal classification model through the incremental learning mechanism.

[0038] In the embodiments of the present disclosure, in order to enable the illegal classification model to continuously adapt to the newly appeared illegal content type, and avoid the identification failure caused by the model being unable to respond to new illegal features in time, step 104 first obtains the new illegal label generated in the foregoing step, and then updates the illegal classification model through the incremental learning mechanism. Among them, obtaining the new illegal label needs to confirm its corresponding illegal semantic attribute and related text features, to ensure that the new label can accurately represent the new type of illegal content; the incremental learning mechanism needs to have the ability to adjust the model parameters using the sample data associated with the new illegal label (including new illegal samples and normal samples) without retraining the full amount of historical data, so as to integrate the identification ability of the new illegal type on the basis of retaining the identification performance of the model for the existing illegal types, and realize the efficient iteration of the model. As an implementation, the new illegal samples and normal samples corresponding to the new illegal label can be collected to form an incremental data set, the illegal classification model is trained using the incremental data set, and the weights and parameters of the model are updated, so that the model can identify the illegal categories corresponding to the new illegal label.

[0039] The model is updated by a new violation label through an incremental learning mechanism, which solves the problem in the prior art that the model needs to be retrained by full data to identify new violation types, which has high update cost and poor timeliness, and achieves the technical effects of improving the dynamic adaptability of the violation classification model, reducing the complexity and cost of model updating, and ensuring that the model can maintain high identification accuracy for new and old violation types.

[0040] The content review method provided by the present disclosure generates a text vector by performing multi-level feature extraction on the original text, clusters the text vector by using a self-adaptive hierarchical clustering algorithm, extracts key text features based on the clustering results to generate hierarchical violation labels, and updates the violation classification model by using new violation labels through an incremental learning mechanism. Therefore, the problems in the prior art that key word filtering cannot understand the semantics and is difficult to identify text variants, traditional hierarchical clustering has high computational complexity and cannot adapt to large-scale data, machine learning classification models lack a dynamic updating mechanism and lead to missed detection of new violation content, and models are difficult to continuously adapt to new review requirements can be solved, and the technical effects of improving the semantic identification accuracy of violation content, improving the real-time performance of large-scale text review, automatically discovering new violation categories and reducing missed detection, and enhancing the scalability and dynamic adaptability of the content review system are achieved.

[0041] Under the technical solution framework disclosed in step 101, multi-level feature extraction is performed on the original text to generate a text vector of the original text, which is further specified as follows: character-level vector encoding is performed on the original text to identify shape and pronunciation-related variants; word-level vector encoding is performed on the original text to capture context semantic features; phrase-level vector encoding is performed on the original text to extract word fixed collocation patterns; and the character-level, word-level and phrase-level vector encodings are stacked to form the text vector of the original text.

[0042] Specifically, in the process of multi-level feature extraction of the original text to generate the text vector, first, character-level vector encoding is performed on the original text, and a pre-trained language model is used to vectorize the original text character by character. Through this encoding method, the association between the text bottom form and pronunciation can be established, thereby effectively identifying text variants such as homophonic variants like “river crab” instead of “harmony”, and homographic characters that deliberately avoid review; then, word-level vector encoding is performed, the original text is first segmented into meaningful lexical units using a segmentation tool adapted to the target language, and then a bidirectional LSTM network is used to analyze the segmented lexical sequence. This network can capture the combination relationship between words from both the front and back texts, and then accurately identify content such as “spinach” representing vegetables in the daily context, but referring to gambling in a specific context, which depends on the context to determine whether it is illegal or not, fully capturing the contextual semantic features of the text; then, phrase-level vector encoding is performed, and a convolutional neural network is used to process text segments within a fixed window in the original text, extracting n-gram features therein to efficiently find common fixed collocation patterns in illegal content; finally, the character-level vector encoding results are summed according to the number n1 of characters in the text, the word-level vector encoding results are summed according to the number n2 of words in the text, and the phrase-level vector encoding results are summed according to the number n3 of phrases in the text. The sum of the three levels is added, i.e. text vector = character vector sum + word vector sum + phrase vector sum, and finally a text vector is formed that can fully represent the semantics of the original text.

[0043] Through hierarchical and targeted encoding methods, the accurate capture of text variants, context-dependent illegal content, and fixed collocation illegal patterns is achieved. After vector superposition, a semantic complete text vector is formed, effectively avoiding the problem of incomplete coverage of text semantics by single-level feature extraction, providing more accurate and comprehensive feature support for subsequent clustering and illegal identification, and significantly improving the recognition ability of complex illegal text.

[0044] Under the technical scheme framework disclosed in step 102, an adaptive hierarchical clustering algorithm is used to cluster the text vector, which is further specified as: based on the density distribution of the text vector, dynamically calculating the effective neighborhood range of each data point; based on the preset geometric distance threshold and vector similarity threshold, the text vector is clustered and merged to generate a clustering cluster.

[0045] Specifically, in the process of clustering the text vectors by using the adaptive hierarchical clustering algorithm, firstly, global density analysis is performed on all text vectors to be clustered, and the distribution density of the region where each data point is located is determined by counting the number of other data points within a specific range in the vector space; then, the effective neighborhood range of each data point is dynamically calculated according to the distribution density - if the data point is located in a core region with higher density, the effective neighborhood range is set to a smaller value to avoid irrelevant data points from being mixed in; if it is located in an edge region with lower density, the effective neighborhood range is appropriately expanded to ensure that semantically related sparse data points are included. After the effective neighborhood calculation is completed, the clustering and merging link is entered: the geometric distance threshold (such as the cosine distance threshold or the Euclidean distance threshold) is set in advance according to the feature distribution of the text vectors (such as the statistical distance of historical violation and compliance text vectors in space); at the same time, the vector similarity threshold is set according to the semantic association requirement (combined with the judgment standard of semantic matching in the past review); for any two text vectors or initially formed micro clusters, the geometric distance and vector similarity of the two are calculated respectively, and only when the geometric distance is less than the preset geometric distance threshold and the vector similarity is greater than the preset vector similarity threshold, the two text vectors (or micro clusters) are merged, gradually aggregated to form a clustering cluster with similar semantic and spatial features, during which redundant clusters with insufficient semantic association after merging are removed, and finally the accurate clustering result is obtained.

[0046] By dynamically calculating the effective neighborhood range, the problem of non-uniform distribution of text vectors that cannot be adapted by traditional fixed neighborhood clustering is solved; combined with double threshold clustering and merging, the clustering result has both spatial proximity and semantic relevance, which not only improves the clustering accuracy, but also reduces the invalid merging operation, effectively reduces the computing cost of large-scale text vector clustering, and adapts to the real-time review scene requirements.

[0047] Under the technical scheme framework disclosed in step 103, based on the clustering result, the key text features are extracted, which are further specified as follows: the weights of the words and phrases in the clustering cluster are calculated, and the weights are based on the frequency, inverse document frequency and distance from the cluster center; the highest weight of the preset number of words and phrases is selected as the key text features.

[0048] Specifically, in the process of extracting key text features based on the clustering results, first, the word and phrase splitting is performed on all the texts in each cluster, and the occurrence frequency n word of each word and the occurrence frequency n phrase of each phrase in the cluster are counted by traversing each text data in the cluster, that is, n word is the total frequency of the occurrence of the word in all the texts in the current cluster, and n phrase is the total frequency of the occurrence of the phrase in all the texts in the current cluster. Then, the inverse document frequency idf word and idf phrase of the word and the phrase are calculated, wherein the calculation of idf word is based on the total amount of the to-be-reviewed text set, and specifically, idf word = log(total number of to-be-reviewed text set / number of texts containing the word in the to-be-reviewed text set), and the calculation of idf phrase is the same, that is, idf phrase = log(total number of to-be-reviewed text set / number of texts containing the phrase in the to-be-reviewed text set), so as to reflect the distinguishing degree of the word and the phrase in the total text. Then, the distance between the word, the phrase and the cluster center is calculated, the cluster center vector is obtained by averaging all the text vectors in the cluster, and then the cosine distance formula is used to calculate the cosine distance distance(word, center) between the vector of each word (generated by the word embedding technology in the aforementioned text preprocessing stage) and the cluster center vector, and the cosine distance distance(phrase, center) between the vector of each phrase (generated by the phrase-level vector encoding) and the cluster center vector. Then, the weight is calculated according to the above parameters, the word weight q word is calculated according to the formula q word = n word * idf word / distance(word, center), and the phrase weight q phrase is calculated according to the formula q phrase = n phrase * idf phrase / distance(phrase, center); after the weight calculation is completed, the weights of all the words and the weights of all the phrases in the cluster are sorted in descending order, and then according to the preset number (which can be adjusted according to the characteristics of the violation category, such as 20 for the political category cluster and 15 for the pornography category cluster), the words and phrases with the top preset number of weights are selected as the key text features corresponding to the cluster.

[0049] By fusing the occurrence frequency, the inverse document frequency and the cluster center distance to calculate the weight, the redundant feature interference caused by relying only on the frequency is avoided, and the features deviating from the core semantics of the cluster are excluded, and the preset number of features with the highest weight can accurately lock the core semantic information of the cluster, providing a reliable basis for subsequent generation of accurate hierarchical violation labels, and effectively improving the accuracy and pertinence of key text feature extraction.

[0050] Under the technical solution framework disclosed in step 103, a hierarchical label structure describing the illegal content is generated, which is further specified as follows: the key features are input into a pre-trained language model to generate three-level labels including illegal types, illegal topics and illegal entities.

[0051] Specifically, in the process of generating a hierarchical label structure describing the illegal content based on the clustering results, the aforementioned extracted key text features (i.e., the top pre-set number of words and phrases with the highest weights) and the pre-set number of texts closest to the cluster center in the cluster (used to provide semantic context support) are input into a pre-trained language model together. The pre-trained language model has been fine-tuned for content review scenarios based on massive illegal and compliant text data, and can accurately capture semantic associations and hierarchical relationships. The model first performs comprehensive semantic analysis on the key features and context texts to generate first-level coarse-grained illegal type labels, which cover multiple illegal categories and clearly indicate the main illegal category to which the cluster belongs. Then, based on the first-level labels, the semantic details are further explored to generate second-level specific illegal topic labels. Finally, specific pointing information in the text is further extracted to generate third-level illegal entity or expression method labels, such as "distorting the history of the Anti-Japanese War" corresponding to the "historical nihilism" topic. At the same time, the pre-trained language model also synchronously generates explanations for each level of label, including typical illegal examples taken from the texts in the cluster, semantic differentiation relationships between labels, and core judgment basis keyword lists, forming a complete three-level hierarchical label structure.

[0052] The specific embodiment uses a pre-trained language model fine-tuned for scenarios to generate three-level labels in combination with key features and context, which not only ensures the clarity and semantic coherence of the label hierarchy, but also improves the accuracy of the labels in pointing to illegal content. The synchronous generation of explanations also provides intuitive references for manual review, effectively solving the problems of ambiguous label hierarchy and unclear semantic definition in traditional label structures, and significantly improving the efficiency and accuracy of illegal content classification.

[0053] Under the technical solution framework disclosed in step 104, the generated new illegal labels are obtained, and the illegal classification model is updated through an incremental learning mechanism, which is further specified as follows: new illegal samples and normal samples are collected to construct an incremental training set; and the incremental training set is used to perform incremental learning training on the illegal classification model.

[0054] Specifically, in the process of obtaining the generated new violation label and updating the violation classification model through the incremental learning mechanism, first, sample collection is carried out: from the text clustering cluster corresponding to the generated new violation label, the text that can typically reflect the semantic features of the new violation label is selected as a new violation sample, and when the text is selected, the text that is close to the cluster center vector in the clustering cluster and contains the core judgment keyword of the new violation label is preferentially selected to ensure the representativeness of the new violation sample; at the same time, the compliance text irrelevant to the semantic of the new violation label is collected as a normal sample, and the normal sample needs to cover different scenes such as daily communication, legal information and regular commodity introduction to avoid the single sample scene leading to insufficient generalization ability of the model. After the new violation sample and the normal sample are collected, the text preprocessing operation (including removing non-text content, word segmentation and vector encoding) consistent with step 101 is performed on the collected new violation sample and the normal sample, the data format is unified and invalid data is removed, and then an incremental training set is constructed according to a preset ratio (for example, the ratio of the new violation sample to the normal sample is 1:2, which can be dynamically adjusted according to the actual distribution of the two types of content in the actual audit scene). Subsequently, incremental learning training is performed: the incremental training set is input into the current violation classification model, an incremental learning strategy based on parameter fine-tuning is adopted, only the network layer parameters related to the identification of new violation features in the model (such as the weight parameters of the output layer of the model corresponding to the new violation category and the convolution kernel parameters of the intermediate feature extraction layer related to the new violation semantic) are updated, and there is no need to load the historical full training data for retraining; during the training process, a low learning rate (such as 1e-5) and a reasonable training round (such as 5-10 rounds) are set, and the objective function is gradually optimized by the gradient descent method, so that the model can fully learn the features of the new violation sample while avoiding the model forgetting the identification ability of the original violation category due to the large parameter update amplitude, and finally the parameter iteration and performance update of the violation classification model are completed.

[0055] By collecting samples for constructing the incremental training set, the semantic matching degree of the training data and the new violation label is ensured, and high-quality data support is provided for the model to learn new violation features; by using the incremental learning training method of parameter fine-tuning, the high calculation cost and long time consumption of full data retraining are avoided, and the model "catastrophic forgetting" is effectively prevented, so that the model can quickly master the identification ability of new violation content while maintaining the identification accuracy of the original violation category, and the efficiency and practicality of dynamic updating of the violation classification model are significantly improved.

[0056] Under the technical solution framework disclosed in the foregoing embodiments, the embodiments of the present application further comprise: verifying the sensitive topics in the clustering results to confirm their violation; adding the confirmed violation categories to the violation label library and updating the hierarchical label structure.

[0057] Specifically, in the process of verifying the sensitive topics in the clustering results and updating the violation label library and the hierarchical label structure, first, based on the clustering clusters obtained by the foregoing hierarchical clustering and the preset sensitive word library (the sensitive word library covers the core sensitive words, phrases and semantic expressions under each violation category that have been explicitly defined), sensitive topics in which the key text features in the clustering clusters and the words in the sensitive word library have semantic association or contain sensitive words are screened out - for example, if a clustering cluster key feature contains "distort revolutionary history" related words and matches the "historical nihilism" category sensitive word in the sensitive word library, the topic corresponding to the clustering cluster is marked as a sensitive topic to be verified. Subsequently, a verification process is started: on the one hand, a professional reviewer with content review qualifications reviews representative texts (texts with a closer distance to the cluster center) in the clustering cluster one by one in combination with the hierarchical preliminary label and label explanation (including typical example sentences, judgment keywords) corresponding to the sensitive topic to be verified, and judges whether the text contains substantive violation semantics; on the other hand, a rule-based automatic verification mechanism is simultaneously started, which verifies the manual review result by comparing the text of the sensitive topic to be verified with the preset violation judgment rule library (including semantic logic rules and context association rules), and assists in verifying the manual review result. If the manual review result and the automatic verification result are consistent, the violation of the topic is confirmed; if the results are different, secondary manual review is started to determine the final violation judgment result.

[0058] Through the "screening - double-track verification - accurate updating" process, the misjudgment or omission of sensitive topics caused by relying only on automatic screening is avoided, and the accuracy of violation judgment is ensured through the verification method combining manual and automatic verification; at the same time, the confirmed violation category specification is added to the label library and the hierarchical structure is updated, ensuring the timeliness of the violation label library and the integrity of the hierarchical label structure, providing accurate and latest violation category basis for subsequent violation classification model training and automatic review, effectively improving the reliability and consistency of the content review system in judging violation content.

[0059] It should be noted that the embodiments of the present disclosure can include a plurality of steps, which are numbered for the convenience of description, but these numbers do not limit the execution time slots and execution order between the steps; the steps can be implemented in any order, and the embodiments of the present disclosure do not limit this.

[0060] Corresponding to the content review method described above, the present disclosure also proposes a content review device. Since the device embodiments of the present disclosure correspond to the method embodiments described above, for details not disclosed in the device embodiments, reference can be made to the method embodiments described above, which will not be described in detail in the present disclosure.

[0061] Figure 2 A structural schematic diagram of a content review device provided by an embodiment of the present disclosure is shown in Figure 2 , which includes: The extraction unit 21 is configured to perform multi-level feature extraction on the original text to generate a text vector of the original text. The clustering unit 22 is configured to perform clustering on the text vector by using an adaptive hierarchical clustering algorithm. The generation unit 23 is configured to extract key text features based on the clustering result and generate a hierarchical label structure describing the violation content. The first updating unit 24 is configured to obtain the generated new violation label and update the violation classification model by using an incremental learning mechanism.

[0062] The content review device provided by the present disclosure can perform multi-level feature extraction on the original text to generate a text vector, perform clustering on the text vector by using an adaptive hierarchical clustering algorithm, extract key text features based on the clustering result to generate a hierarchical violation label, and update the violation classification model by using a new violation label through an incremental learning mechanism. Therefore, the problems in the prior art, such as difficulty in identifying text variants due to insufficient semantic understanding of keyword filtering, high computational complexity of traditional hierarchical clustering that cannot adapt to large-scale data, new violation content missing due to lack of dynamic updating mechanism of machine learning classification model, and difficulty in continuously adapting to new review requirements of the model, can be solved. The technical effects of improving semantic recognition accuracy of violation content, improving real-time performance of large-scale text review, automatically discovering new violation categories and reducing missing, and enhancing scalability and dynamic adaptability of the content review system can be achieved.

[0063] Further, in a possible implementation manner of the present embodiment, the extraction unit 21 is further configured to: perform character-level vector encoding on the original text to identify variants related to character shape and pronunciation; perform word-level vector encoding on the original text to capture context semantic features; perform phrase-level vector encoding on the original text to extract fixed collocation patterns of words; superimpose the character-level, word-level, and phrase-level vector encodings to form the text vector of the original text.

[0064] Further, in a possible implementation manner of the present embodiment, the clustering unit 22 is further configured to: dynamically calculate an effective neighborhood range of each data point based on the density distribution of the text vector; perform clustering and merging on the text vector based on a preset geometric distance threshold and a vector similarity threshold to generate a clustering cluster.

[0065] Further, in a possible implementation manner of the present embodiment, the generation unit 23 is further configured to: calculate weights of the words and phrases in the clustering cluster, the weights being based on frequency of occurrence, inverse document frequency, and distance from the cluster center; select a preset number of words and phrases with the highest weights as the key text features.

[0066] Further, in a possible implementation of the embodiment, the generating unit 23 is further configured to: input the key features into the pre-trained language model to generate a three-level label containing the violation type, the violation theme and the violation entity.

[0067] Further, in a possible implementation of the embodiment, the first updating unit 24 is further configured to: collect new violation samples and normal samples to construct an incremental training set; perform incremental learning training on the violation classification model using the incremental training set.

[0068] Further, in a possible implementation of the embodiment, as shown in Figure 2 the embodiment further comprises: a second updating unit 25 configured to verify the sensitive theme in the clustering result and confirm its violation; add the confirmed violation category to the violation label library and update the hierarchical label structure.

[0069] It should be noted that the foregoing explanation and description of the method embodiments are also applicable to the device of the present embodiment, and the principle is the same, which will not be limited in the present embodiment.

[0070] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0071] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0072] As Figure 3As shown, the electronic device 300 includes a computing unit 301 that can perform various appropriate actions and processes in accordance with a computer program stored in a ROM (Read-Only Memory) 302 or a computer program loaded into a RAM (Random Access Memory) 303 from a storage unit 308. Various programs and data required for the operation of the electronic device 300 can also be stored in the RAM 303. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An I / O (Input / Output) interface 305 is also connected to the bus 304.

[0073] Various components in the electronic device 300 are connected to the I / O interface 305, including an input unit 306 such as a keyboard, a mouse, and the like, an output unit 307 such as various types of displays, a speaker, and the like, a storage unit 308 such as a magnetic disk, an optical disk, and the like, and a communication unit 309 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0074] The computing unit 301 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, and the like. The computing unit 301 performs various methods and processes described above, such as the content review method. For example, in some embodiments, the content review method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 301 can be configured to perform the aforementioned content review method by any other appropriate means, such as by means of firmware.

[0075] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a Field Programmable Gate Array (FPGA), an Application-Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a System on a Chip (SOC), a Complex Programmable Logic Device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0076] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general or special purpose computer, such that the program code, when executed by the processor or controller, causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can execute entirely on a machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0077] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory), or flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0078] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0079] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a blockchain network.

[0080] The computer system can include clients and servers. This relationship can be between a client and a server that are typically distant from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0081] It should be noted that artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of people, both hardware and software technologies. Artificial intelligence hardware technology generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing, etc.; artificial intelligence software technology mainly includes computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, etc. several major directions.

[0082] The first, second, and the like various numerical numbers involved in the present disclosure are only for the convenience of differentiation in the description, and do not limit the scope of the embodiments of the present disclosure, nor represent the order of precedence.

[0083] At least one of the present disclosure can also be described as one or more, and the plurality can be two, three, four or more, which is not limited by the present disclosure. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D". The technical features described by "first", "second", "third", "A", "B", "C" and "D" have no order or size order.

[0084] It should be understood that the steps shown above can be reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, which is not limited herein.

[0085] The above detailed description does not limit the scope of the disclosure. Various modifications, combinations, sub-combinations and alternatives can be made to the detailed description. Any modification, equivalent replacement and improvement etc. made within the spirit and principle of the disclosure shall be included in the scope of the disclosure.

Claims

1. A content moderation method, characterized in that, include: Multi-level feature extraction is performed on the original text to generate a text vector of the original text; The text vectors are clustered using an adaptive hierarchical clustering algorithm; Key text features are extracted based on clustering results, and a hierarchical tag structure describing the illegal content is generated. Obtain the newly generated violation labels and update the violation classification model through an incremental learning mechanism.

2. The method according to claim 1, characterized in that, The step of extracting multi-level features from the original text to generate a text vector of the original text includes: The original text is subjected to character-level vector encoding to identify glyph and pronunciation-related variants; The original text is subjected to word-level vector encoding to capture contextual semantic features; The original text is subjected to phrase-level vector encoding to extract fixed word collocation patterns; The text vector of the original text is formed by superimposing character-level, word-level, and phrase-level vector codes.

3. The method according to claim 1, characterized in that, The step of clustering the text vectors using an adaptive hierarchical clustering algorithm includes: Based on the density distribution of the text vector, the effective neighborhood range of each data point is dynamically calculated; Based on preset geometric distance thresholds and vector similarity thresholds, the text vectors are clustered and merged to generate clusters.

4. The method according to claim 1, characterized in that, The extraction of key text features based on clustering results includes: Calculate the weights of words and phrases in the clusters, based on occurrence frequency, inverse document frequency, and distance from the cluster center; The words and phrases with the highest weights are selected as the key text features.

5. The method according to claim 4, characterized in that, The generation of a hierarchical tag structure describing the violation content includes: Key features are input into a pre-trained language model to generate three-level labels containing violation type, violation topic, and violation entity.

6. The method according to claim 1, characterized in that, The process of obtaining the newly generated violation labels and updating the violation classification model through an incremental learning mechanism includes: Collect new violation samples and normal samples to construct an incremental training set; The violation classification model is trained incrementally using the incremental training set.

7. The method according to claim 1, characterized in that, Also includes: Verify sensitive topics in the clustering results to confirm their irregularities; Add the confirmed violation categories to the violation tag library and update the hierarchical tag structure.

8. A content moderation device, characterized in that, include: The extraction unit is used to extract multi-level features from the original text and generate a text vector of the original text. Clustering unit, used to cluster the text vectors using an adaptive hierarchical clustering algorithm; The generation unit is used to extract key text features based on clustering results and generate a hierarchical label structure describing the illegal content. The first update unit is used to obtain the newly generated violation labels and update the violation classification model through an incremental learning mechanism.

9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images