Chinese bad speech detection method and system

Abstract

The invention provides a Chinese bad speech detection method and system, and relates to the technical field of speech detection, and the method comprises the steps: obtaining an initial tweet data set containing a bad speech; performing preprocessing including data cleaning and formatting on the initial tweet data set; classifying and marking the preprocessed initial tweet data set by utilizing a multi-model consistency voting strategy to obtain a Chinese bad speech data set; constructing a Chinese bad speech detection model; inputting the Chinese bad speech data set into a Chinese bad speech detection model for training; acquiring a real-time Chinese bad speech data set; and inputting the Chinese bad speech data set into the trained Chinese bad speech detection model, and outputting a Chinese bad speech detection result. According to the method, the detection effect of the Chinese bad speech is improved, and the model can comprehensively capture the essential characteristics of the input sample from multiple angles.

Description

Technical Field

[0001] The present invention relates to the technical field of speech detection, and in particular to a method and system for detecting bad Chinese speech. Background Art

[0002] The Chinese bad speech detection method is mainly used to identify and filter Chinese texts containing inappropriate content. It combines keyword matching, machine learning classifiers, natural language processing technology, and multimodal information analysis. It aims to accurately identify bad speech by understanding the semantics and context of the text, as well as possible image and audio information. With the development of technology, these detection technologies are becoming more and more intelligent and automated, effectively helping to manage and filter bad information in the network environment.

[0003] The rampant spread of negative speech not only disrupts the public order in cyberspace, but also stimulates social emotional confrontation and division, erodes the basis for rational discussion among the people, and poses a potential threat to social stability and harmonious development. In the digital age, Chinese negative speech detection methods are key tools for monitoring and identifying such speech.

[0004] However, existing methods for detecting Chinese offensive speech mainly rely on keyword matching, which cannot effectively handle context and implicit meaning, and may lead to misjudgment and omission. Traditional models often lack an understanding of the deep semantics of the text and have difficulty identifying more subtle or variant forms of offensive content, resulting in low efficiency in detecting Chinese offensive speech. Summary of the invention

[0005] In order to solve the technical problems that the existing methods for detecting bad Chinese speech mainly rely on keyword matching, cannot effectively process context and implicit meaning, may cause misjudgment and omission, and traditional models often lack understanding of the deep semantics of the text, and have difficulty in identifying more subtle or variant forms of bad content, resulting in low efficiency in detecting bad Chinese speech, the present invention provides a method and system for detecting bad Chinese speech.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect:

[0008] The embodiment of the present invention provides a method for detecting bad Chinese speech, comprising:

[0009] S1: Get the initial tweet dataset containing bad words;

[0010] S2: Preprocess the initial tweet dataset;

[0011] S3: Use the multi-model consensus voting strategy to classify and annotate the preprocessed initial tweet dataset to obtain the Chinese bad speech dataset;

[0012] S4: constructing a Chinese bad speech detection model, wherein the Chinese bad speech detection model includes a data enhancement module, an R-Drop regularization module, a dual-channel classification task module and a contrastive learning module connected in sequence;

[0013] S5: Input the Chinese bad speech dataset into the Chinese bad speech detection model for training;

[0014] S6: Obtain a real-time dataset of Chinese offensive speech;

[0015] S7: Input the Chinese bad speech data set into the trained Chinese bad speech detection model, and output the Chinese bad speech detection result.

[0016] Second aspect:

[0017] The embodiment of the present invention provides a Chinese bad speech detection system, comprising:

[0018] processor;

[0019] A memory having computer-readable instructions stored therein, which, when executed by a processor, implements the method for detecting bad Chinese speech as described in the first aspect.

[0020] The third aspect:

[0021] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method for detecting bad Chinese speech according to the first aspect is implemented.

[0022] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0023] In the present invention, a multi-model consistency voting strategy is utilized to improve the accuracy and reliability of detection, help balance the deviations of different models, and reduce the misjudgment that may be caused by a single model. The data enhancement module can expand the diversity of training samples. The R-Drop regularization module can achieve better generalization on the diversified data of the model and reduce overfitting. The dual-channel classification task module can effectively capture the subtle differences in bad speech, deeply understand the differences from different perspectives, enhance the robustness of the model, and improve the final classification effect, ensuring that the model can comprehensively capture the essential characteristics of the input samples from multiple angles. Through the comparative learning module, the subtle differences between bad speech and normal speech can be better learned and distinguished, thereby improving the sensitivity and accuracy of the overall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic diagram of a flow chart of a method for detecting bad Chinese speech provided by an embodiment of the present invention;

[0026] Figure 2 A schematic diagram of the structure of a Chinese bad speech detection model provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of the structure of a Chinese bad speech detection system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0030] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0031] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0032] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0033] Reference Manual Attached Figure 1 , which shows a flow chart of a method for detecting bad Chinese speech provided by an embodiment of the present invention.

[0034] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a Chinese bad speech detection model provided by an embodiment of the present invention.

[0035] like Figure 2 The Chinese bad speech detection model adopts a dual-channel BERT architecture. The two paths share parameters and perform feature extraction on the input text respectively to generate different representations. The contrast loss is calculated through the contrast learning module to improve the model's ability to recognize differences between features. At the same time, the two channels each generate classification predictions and generate classification losses. Finally, by combining the contrast loss and classification loss, the model is optimized together to more accurately classify and detect bad speech.

[0036] The embodiment of the present invention provides a method for detecting bad Chinese speech, the method comprising:

[0037] S1: Obtain an initial dataset of tweets containing offensive speech.

[0038] In a possible implementation, by obtaining an initial tweet dataset containing negative speech, the characteristics of negative speech in actual social networks can be better reflected.

[0039] In a possible implementation, S1 specifically includes:

[0040] S101: Based on a preset list of sensitive words for negative speech, search through the Twitter API to filter out tweets containing sensitive words.

[0041] Among them, Twitter API is an application programming interface provided by the Twitter platform, which allows programs to interact with Twitter data, such as searching tweets, obtaining user information, etc.

[0042] It should be noted that by searching through the Twitter API based on a list of sensitive words, tweets containing sensitive content can be quickly screened out, potential negative speech can be quickly located, and a lot of time and human resources can be saved for subsequent data annotation and model training, while ensuring that the preliminary data is targeted and reducing the noise of irrelevant data.

[0043] S102: Perform preliminary manual labeling on the tweets to obtain labeled tweets.

[0044] Specifically, manual annotation can ensure that the meaning of sensitive words in specific contexts is correctly understood, avoiding misjudgments and omissions, thereby providing a high-quality data foundation for subsequent model training and improving the reliability and accuracy of the model when processing complex semantics and implicit expressions.

[0045] S103: Record the usernames of the authors of the annotated tweets to form a list of high-risk users of harmful speech.

[0046] S104: Based on the list of high-risk users of bad speech, historical tweets of high-risk users of bad speech are captured through Twitter API to form initial tweet data.

[0047] S2: Preprocess the initial tweet dataset.

[0048] It should be noted that preprocessing the initial tweet dataset can clean up noisy data, improve data consistency and normalization, thereby enhancing the model's ability to extract effective information and reducing misjudgments caused by differences in data quality.

[0049] In a possible implementation manner, S2 is specifically:

[0050] The initial tweet dataset is preprocessed including data cleaning and formatting.

[0051] Specifically, preprocessing including data cleaning and formatting includes removing non-Chinese content in tweets, including English characters, numbers, symbols, etc., retaining only Chinese characters, cleaning up irrelevant information in tweets, including deleting URL links, @user information and emoticons, converting traditional Chinese in tweets to simplified Chinese, unifying the language format, filtering short text content, and retaining only tweets with a length greater than or equal to 5 characters to form a structured high-quality tweet dataset.

[0052] S3: Use the multi-model consensus voting strategy to classify and annotate the preprocessed initial tweet dataset to obtain the Chinese bad speech dataset.

[0053] Among them, the multi-model consistency voting strategy refers to using multiple different models to predict data at the same time, voting through the prediction results of each model, and when most models give the same classification result, it is used as the final classification result to improve the accuracy and stability of the judgment.

[0054] It should be noted that using a multi-model consistency voting strategy to classify and label the preprocessed tweet dataset can effectively reduce the bias caused by a single model and improve the reliability of the classification results.

[0055] In a possible implementation, S3 specifically includes:

[0056] S301: Divide the preprocessed initial tweet dataset into a first dataset and a second dataset.

[0057] S302: Manually annotate the first data set to form an initial annotated data set.

[0058] S303: Select multiple Chinese pre-training models, where the Chinese pre-training models include a BERT-Base model, a RoBERTa-wwm-ext model, a MacBERT model, and a Chinese-ELECTRA model.

[0059] Specifically, the BERT-Base model is a pre-trained language model based on the bidirectional Transformer structure, which is used to capture contextual information. The RoBERTa-wwm-ext model is an enhanced version of BERT, which improves language understanding ability through the "full word masking" strategy. The MacBERT model is an improved version of BERT, focusing on the optimization of text matching and text classification tasks. The Chinese-ELECTRA model is a new type of pre-training model that improves training efficiency by replacing traditional generation tasks with discrimination tasks.

[0060] S304: Based on the initial annotated data set, fine-tune each Chinese pre-trained model for the binary classification task.

[0061] S305: Input the second data set into each fine-tuned Chinese pre-training model for automatic classification, and generate the classification result and consistency ratio of the second data set.

[0062] S306: Determine whether the consistency ratio is greater than a preset ratio. If so, output the classification result as an automatic labeling result. Otherwise, manually label the second data set to form a final labeled data set.

[0063] In actual operation, if there are 75% or more consistent prediction results among the four models, the consistent results will be directly used as the annotation results of the tweet. For tweets with consistency lower than 75%, they will be submitted to the manual annotation team for manual annotation to ensure the accuracy of the annotation results.

[0064] S307: Add the final annotated dataset to the initial annotated dataset, and repeat steps S303 to S307 until the high-quality tweet dataset is annotated.

[0065] S308: Outputting the Chinese offensive speech dataset.

[0066] S4: Construct a Chinese bad speech detection model, wherein the Chinese bad speech detection model includes a data enhancement module, an R-Drop regularization module, a dual-channel classification task module and a contrastive learning module connected in sequence.

[0067] It should be noted that by integrating the data enhancement module, R-Drop regularization module, dual-channel classification task module and contrastive learning module, the model is comprehensively improved in terms of sample richness, prediction stability, classification accuracy and feature robustness, effectively enhancing the detection ability and adaptability of bad Chinese speech.

[0068] S5: Input the Chinese bad speech dataset into the Chinese bad speech detection model for training.

[0069] It should be noted that by training the model, the model can fully learn the data features, thereby improving the accuracy and generalization ability of detection.

[0070] In a possible implementation, the data enhancement module includes a BERT unit, and the BERT unit includes a Dropout algorithm.

[0071] Specifically, the BERT unit is responsible for extracting contextual features from the text to improve text understanding and processing capabilities. The Dropout algorithm is a regularization method that randomly discards some neurons during the training process to prevent overfitting and improve the generalization ability of the model. The Dropout algorithm is built into the Bert unit. By randomly "closing" some neurons, the model will not overly rely on certain specific neurons during training, thereby effectively reducing the risk of overfitting. By randomly "closing" neurons, different feature representations will be output for the same sample. The same sample is input into the BERT unit, and the output features are reinforced samples of each other.

[0072] S5 specifically includes:

[0073] S501: Based on the Chinese bad speech dataset, feature extraction is performed through the BERT unit to obtain data features.

[0074] It should be noted that by performing feature extraction on the Chinese offensive speech dataset through the BERT unit, we can deeply explore the contextual information and semantic features of the text, thereby obtaining high-quality data representation.

[0075] In a possible implementation, S501 specifically includes:

[0076] S5011: Load the Chinese bad speech dataset:

[0077]

[0078] Among them, D represents the bad speech dataset, x i represents the i-th input sentence in the bad speech dataset, y i Represents x i The corresponding true label;

[0079] Specifically, y i The value of is a binary category, 0 represents "normal speech" and 1 represents "bad speech".

[0080] S5012: Perform text preprocessing including word segmentation, special marking and conversion to BERT format on the Chinese bad speech dataset to obtain text data, wherein the special marking includes adding a mark [CLS] at the beginning of the sentence in the Chinese bad speech dataset and adding a mark [SEP] at the end of the sentence in the Chinese bad speech dataset.

[0081] S5013: Divide the text data into preset sizes to generate batch data.

[0082] S5014: Input the batch data into the BERT unit and extract the output vector containing the [CLS] tag as the data feature:

[0083]

[0084] Among them, h CLS represents the data characteristics, f BERT Represents the feature extraction function, input_ids represents the vocabulary index sequence of the input sentence, attention_mask represents the valid input token, and token_type_ids represents the distinction between sentence segments.

[0085] S502: Using the Dropout algorithm, perform data enhancement on the feature data to generate positive sample pairs.

[0086] In a possible implementation, S502 specifically includes:

[0087] S5021: Use the Dropout algorithm to double encode the data features to generate the first feature representation and the second feature representation:

[0088]

[0089] in, represents the first feature representation, represents the second feature representation, x represents the input sentence, represents the first Dropout mask, Represents the second Dropout mask.

[0090] S5022: The first feature representation and the second feature representation are used as different enhanced features of the same sentence in the Chinese bad speech dataset to form a positive sample pair for contrastive learning, wherein the first feature representation and the second feature representation are correlated.

[0091] S503: Input the positive sample pairs into the classifier of the R-Drop regularization module and output the predicted distribution of the positive sample pairs:

[0092]

[0093] in, represents the first predictive distribution, represents the normalized exponential function, represents the second prediction distribution, W represents the weight matrix of the classifier, b represents the bias term of the classifier, represents the first feature representation, Represents the second feature representation.

[0094] S504: Calculate KL divergence loss and cross entropy loss according to the predicted distribution.

[0095] Among them, KL divergence is an indicator that measures the difference between two probability distributions, which is used to evaluate the degree of match between the model prediction distribution and the target distribution. Cross entropy loss is a loss function that measures the difference between the predicted probability distribution and the true distribution (label), which is often used in classification tasks.

[0096] It should be noted that by combining KL divergence loss and cross entropy loss, the accuracy and consistency of the predicted distribution can be optimized simultaneously, and the classification stability and robustness of the model can be enhanced.

[0097] S505: Input the positive sample pair into the dual-channel classification task module to generate the final classification result.

[0098] In a possible implementation manner, S505 specifically includes:

[0099] S5051: Input the positive sample pair into the dual-channel classification task module to generate a first classification probability distribution and a second classification probability distribution.

[0100]

[0101] in, represents the first classification probability distribution, represents the second classification probability distribution, W1 and W2 represent the weight matrices of the first channel and the second channel classifiers respectively, and b1 and b2 represent the bias items of the first channel and the second channel classifiers respectively.

[0102] S5052: Fusion of the first classification probability distribution and the second classification probability distribution to generate the final classification result:

[0103]

[0104] in, represents the final classification result of the i-th category, represents the first classification probability distribution of the i-th category, represents the second classification probability distribution of the i-th category, Represents the final classification result of the jth category.

[0105] S506: Generate contrastive learning loss through contrastive learning module:

[0106]

[0107] L cl represents contrastive learning loss, sim represents feature similarity, i=1,2,···,n, n represents the number of input sentences involved in loss calculation in the Chinese bad speech dataset, τ represents temperature hyperparameter, J=1,2,···,N, N represents the total number of input sentences in the Chinese bad speech dataset, h j Represents negative samples.

[0108] S507: Combine cross entropy loss, KL divergence loss and contrastive learning loss to construct the total loss function:

[0109]

[0110] Among them, L total Represents the total loss function, L kl represents the KL divergence loss, L ce Denotes the cross entropy loss Represents a hyperparameter.

[0111] In a possible implementation, the KL divergence loss is specifically:

[0112]

[0113] Among them, L kl represents the KL divergence loss, D KL represents the KL divergence, i=1,2,···,n, n represents the number of input sentences participating in the loss calculation in the Chinese bad speech dataset, P(y) represents the predicted probability value of probability distribution P on category y, and Q(y) represents the predicted probability value of probability distribution Q on category y.

[0114] The cross entropy loss is specifically:

[0115]

[0116] Among them, L ce represents the cross entropy loss, m represents, x i represents the i-th input sentence in the Chinese bad speech dataset, y i Represents x iThe corresponding true label, i=1,2,···,n, where n represents the number of input sentences involved in the loss calculation in the Chinese bad speech dataset.

[0117] S508: Using a gradient descent optimization algorithm to adjust the parameters of the Chinese bad speech detection model until the total loss function value is less than a preset loss function value.

[0118] S6: Obtain a real-time dataset of offensive Chinese speech.

[0119] It should be noted that by obtaining the latest Chinese negative speech data in real time, we can ensure that the model can adapt to the dynamically changing language environment and improve the timeliness and coverage of detection.

[0120] S7: Input the Chinese bad speech data set into the trained Chinese bad speech detection model, and output the Chinese bad speech detection result.

[0121] The present invention performs BERT encoding twice on the same sentence, and each encoding uses Dropout randomness to generate different sentence embeddings as positive sample pairs for contrastive learning to achieve data-level enhancement. During the training process, the R-Drop strategy is adopted to effectively utilize the randomness of Dropout by minimizing the bidirectional KL divergence between the model output distributions after two independent Dropout operations, and significantly enhance the robustness and generalization ability of the model. By designing dual-channel classification task constrained feature learning, it is ensured that the model can learn consistent and effective classification information from different feature representations. A contrastive learning mechanism is introduced to bring positive sample pairs closer in the feature space and push other negative samples away, prompting the model to learn more essential and discriminative feature representations.

[0122] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0123] In the present invention, a multi-model consistency voting strategy is utilized to improve the accuracy and reliability of detection, help balance the deviations of different models, and reduce the misjudgment that may be caused by a single model. The data enhancement module can expand the diversity of training samples. The R-Drop regularization module can achieve better generalization on the diversified data of the model and reduce overfitting. The dual-channel classification task module can effectively capture the subtle differences in bad speech, deeply understand the differences from different perspectives, enhance the robustness of the model, and improve the final classification effect, ensuring that the model can comprehensively capture the essential characteristics of the input samples from multiple angles. Through the comparative learning module, the subtle differences between bad speech and normal speech can be better learned and distinguished, thereby improving the sensitivity and accuracy of the overall detection.

[0124] Reference Manual Attached Figure 3, showing a structural schematic diagram of a Chinese bad speech detection system provided by the present invention.

[0125] The present invention further provides a Chinese bad speech detection system 20, which is applied to the above-mentioned Chinese bad speech detection method, comprising:

[0126] Processor 201.

[0127] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201, the Chinese bad speech detection method of the method embodiment is implemented.

[0128] The Chinese bad speech detection system 20 provided by the present invention can execute the above-mentioned Chinese bad speech detection method and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0129] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0130] In the present invention, a multi-model consistency voting strategy is utilized to improve the accuracy and reliability of detection, help balance the deviations of different models, and reduce the misjudgment that may be caused by a single model. The data enhancement module can expand the diversity of training samples. The R-Drop regularization module can achieve better generalization on the diversified data of the model and reduce overfitting. The dual-channel classification task module can effectively capture the subtle differences in bad speech, deeply understand the differences from different perspectives, enhance the robustness of the model, and improve the final classification effect, ensuring that the model can comprehensively capture the essential characteristics of the input samples from multiple angles. Through the comparative learning module, the subtle differences between bad speech and normal speech can be better learned and distinguished, thereby improving the sensitivity and accuracy of the overall detection.

[0131] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0132] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0133] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0134] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0135] In the present invention, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0136] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0143] An embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for detecting bad Chinese speech in the method embodiment is implemented.

[0144] A computer-readable storage medium provided by the present invention can implement the steps and effects of the Chinese bad speech detection method of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0145] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0146] In the present invention, a multi-model consistency voting strategy is utilized to improve the accuracy and reliability of detection, help balance the deviations of different models, and reduce the misjudgment that may be caused by a single model. The data enhancement module can expand the diversity of training samples. The R-Drop regularization module can achieve better generalization on the diversified data of the model and reduce overfitting. The dual-channel classification task module can effectively capture the subtle differences in bad speech, deeply understand the differences from different perspectives, enhance the robustness of the model, and improve the final classification effect, ensuring that the model can comprehensively capture the essential characteristics of the input samples from multiple angles. Through the comparative learning module, the subtle differences between bad speech and normal speech can be better learned and distinguished, thereby improving the sensitivity and accuracy of the overall detection.

[0147] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0148] There are a few points to note:

[0149] (1) The drawings of the embodiments of the present invention only involve structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0150] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0151] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0152] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for detecting bad Chinese speech, characterized in that: include: S1: Get the initial tweet dataset containing bad words; S2: preprocessing the initial tweet dataset; S3: Use the multi-model consensus voting strategy to classify and annotate the preprocessed initial tweet dataset to obtain the Chinese bad speech dataset; S4: constructing a Chinese bad speech detection model, wherein the Chinese bad speech detection model includes a data enhancement module, an R-Drop regularization module, a dual-channel classification task module and a contrastive learning module connected in sequence; S5: inputting the Chinese bad speech data set into the Chinese bad speech detection model for training; S6: Obtain a real-time dataset of Chinese offensive speech; S7: Input the Chinese bad speech data set into the trained Chinese bad speech detection model, and output the Chinese bad speech detection result.

2. The method for detecting bad Chinese speech according to claim 1, characterized in that: The S1 specifically includes: S101: Based on the preset list of sensitive words for bad speech, search through the Twitter API to filter out tweets containing sensitive words; S102: Performing preliminary manual labeling on the tweets to obtain labeled tweets; S103: Record the user name of the author of the marked tweet to form a list of high-risk users of bad speech; S104: According to the list of high-risk users for bad speech, historical tweets of high-risk users for bad speech are captured through Twitter API to form initial tweet data.

3. The method for detecting bad Chinese speech according to claim 1, characterized in that: The S2 is specifically: The initial tweet dataset is preprocessed including data cleaning and formatting.

4. The method for detecting bad Chinese speech according to claim 1, characterized in that: The S3 specifically includes: S301: Divide the preprocessed initial tweet dataset into a first dataset and a second dataset; S302: manually annotating the first data set to form an initial annotated data set; S303: Select multiple Chinese pre-training models, wherein the Chinese pre-training models include a BERT-Base model, a RoBERTa-wwm-ext model, a MacBERT model, and a Chinese-ELECTRA model; S304: fine-tuning each Chinese pre-training model for a binary classification task based on the initial annotated data set; S305: Inputting the second data set into each fine-tuned Chinese pre-training model for automatic classification, and generating a classification result and consistency ratio of the second data set; S306: Determine whether the consistency ratio is greater than a preset ratio, if so, output the classification result as an automatic annotation result, otherwise, manually annotate the second data set to form a final annotated data set; S307: adding the final annotated data set to the initial annotated data set, and repeating steps S303 to S307 until the annotation of the high-quality tweet data set is completed; S308: Outputting the Chinese offensive speech dataset.

5. The method for detecting bad Chinese speech according to claim 4, characterized in that: The data enhancement module includes a BERT unit, and the BERT unit includes a Dropout algorithm; The S5 specifically includes: S501: Based on the Chinese bad speech dataset, feature extraction is performed by the BERT unit to obtain data features; S502: Using the Dropout algorithm, perform data enhancement on the feature data to generate positive sample pairs; S503: Input the positive sample pair into the classifier of the R-Drop regularization module, and output the predicted distribution of the positive sample pair: ； in, represents the first predictive distribution, represents the normalized exponential function, represents the second prediction distribution, W represents the weight matrix of the classifier, b represents the bias term of the classifier, represents the first feature representation, represents the second feature representation; S504: Calculate KL divergence loss and cross entropy loss according to the predicted distribution; S505: Inputting the positive sample pair into the dual-channel classification task module to generate a final classification result; S506: Generate contrastive learning loss through the contrast learning module: ； The L cl represents contrastive learning loss, sim represents feature similarity, i=1,2,···,n, n represents the number of input sentences involved in loss calculation in the Chinese bad speech dataset, τ represents temperature hyperparameter, J=1,2,···,N, N represents the total number of input sentences in the Chinese bad speech dataset, h j represents negative samples; S507: Combining the cross entropy loss, the KL divergence loss and the contrastive learning loss, construct a total loss function: ； Among them, L total Represents the total loss function, L kl represents the KL divergence loss, L ce represents the cross entropy loss, represents a hyperparameter; S508: Using a gradient descent optimization algorithm to adjust the parameters of the Chinese bad speech detection model until the total loss function value is less than a preset loss function value.

6. The method for detecting bad Chinese speech according to claim 5, characterized in that: The S501 specifically includes: S5011: Load the Chinese bad speech dataset: ； Among them, D represents the bad speech dataset, x i represents the i-th input sentence in the bad speech dataset, y i Represents x i The corresponding true label; S5012: performing text preprocessing including word segmentation, special marking and conversion to BERT format on the Chinese bad speech dataset to obtain text data, wherein the special marking includes adding a mark [CLS] at the beginning of a sentence in the Chinese bad speech dataset and adding a mark [SEP] at the end of a sentence in the Chinese bad speech dataset; S5013: Divide the text data into pieces according to a preset size to generate batch data; S5014: Input the batch data into the BERT unit, and extract the output vector containing the [CLS] tag as a data feature: ； Among them, h CLS represents the data characteristics, f BERT Represents the feature extraction function, input_ids represents the vocabulary index sequence of the input sentence, attention_mask represents the valid input token, and token_type_ids represents the distinction between sentence segments.

7. The method for detecting bad Chinese speech according to claim 5, characterized in that: The S502 specifically includes: S5021: Using the Dropout algorithm, double-encode the data features to generate a first feature representation and a second feature representation: ； in, represents the first feature representation, represents the second feature representation, x represents the input sentence, represents the first Dropout mask, Represents the second Dropout mask; S5022: Using the first feature representation and the second feature representation as different enhanced features of the same sentence in the Chinese bad speech dataset to form a positive sample pair for comparative learning, wherein the first feature representation and the second feature representation are correlated.

8. The method for detecting bad Chinese speech according to claim 5, characterized in that: The S505 specifically includes: S5051: Input the positive sample pair into the dual-channel classification task module to generate a first classification probability distribution and a second classification probability distribution: ； in, represents the first classification probability distribution, represents the second classification probability distribution, W1 and W2 represent the weight matrices of the first channel and the second channel classifiers respectively, b1 and b2 represent the bias items of the first channel and the second channel classifiers respectively; S5052: Fusing the first classification probability distribution and the second classification probability distribution to generate a final classification result; ； in, represents the final classification result of the i-th category, represents the first classification probability distribution of the i-th category, represents the second classification probability distribution of the i-th category, Represents the final classification result of the jth category.

9. The method for detecting bad Chinese speech according to claim 5, characterized in that: The KL divergence loss is specifically: ； Among them, L kl represents the KL divergence loss, D KL represents KL divergence, i=1,2,···,n, n represents the number of input sentences involved in the loss calculation in the Chinese bad speech dataset, P(y) represents the predicted probability value of probability distribution P on category y, and Q(y) represents the predicted probability value of probability distribution Q on category y; The cross entropy loss is specifically: ； Among them, L ce represents the cross entropy loss, m represents, x i represents the i-th input sentence in the Chinese bad speech dataset, y i Represents x i The corresponding true label, i=1,2,···,n, where n represents the number of input sentences involved in the loss calculation in the Chinese bad speech dataset.

10. A Chinese bad speech detection system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method for detecting bad Chinese speech as described in any one of claims 1 to 9 is implemented.

Images