Content review automatic punishment method, device and equipment and readable storage medium

Abstract

The application provides a content review automatic punishment method, device and equipment and a readable storage medium. The method provided by the application can analyze the tone emotion information in the content to be reviewed by using a tone emotion analysis model, can effectively improve the accuracy of the review result of the content to be reviewed, can further analyze the text information corresponding to the content to be reviewed by using a text violation detection model, can further fuse and analyze the tone emotion review result and the text information review result of the content to be reviewed by using a multi-modal violation detection model, and can automatically determine and execute the automatic punishment measure for the content to be reviewed after determining that the content to be reviewed contains illegal content, can effectively improve the review accuracy and review efficiency of the content to be reviewed, can effectively reduce the cost of manual review, and can improve the timeliness of punishment for illegal behavior.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information management, and particularly relates to a content review automatic punishment method and device, equipment and a readable storage medium. BACKGROUND

[0002] With the development of the live broadcast industry, the supervision of the pan-entertainment social or live broadcast field tends to be strict. In order to comply with the regulatory requirements, an audio and video live broadcast platform usually needs to monitor the content generated by its users and impose penalties such as user banning on illegal information.

[0003] With the increase in the number of platform users, more audio and video data is generated every day, and tens of thousands to hundreds of thousands of hours of audio and video content can be generated every day. When monitoring the content generated by users, if only artificial monitoring is used, the labor cost is huge and boring. Thanks to the development of deep learning in recent years, a deep learning model can now be used to cooperate with artificial review to review audio and video content. Typically, a deep learning model can first review and confirm illegal information, and then artificial review can determine the audio and video content that is illegal after rechecking, and then corresponding penalties are imposed.

[0004] However, the violation score, recall rate and accuracy rate in the existing review model are not very reasonable in some application scenarios, resulting in a decrease in the final accuracy rate, increasing the cost of artificial review. At the same time, due to the limited human resources, the penalty response is slow, and illegal behavior cannot be stopped in time. SUMMARY

[0005] The present application aims to at least solve one of the above technical defects, and in view of this, the present application provides a content review automatic punishment method, device, equipment and readable storage medium, which is used to solve the technical defect of low review efficiency of voice content in the prior art.

[0006] A content review automatic punishment method, comprising:

[0007] obtaining the content to be reviewed;

[0008] inputting the content to be reviewed into a preset emotional detection model for analysis to obtain an emotional review result output by the preset emotional detection model, wherein the preset emotional detection model is trained with the content to be reviewed as a training sample, and the probability of each tone included in the training of the content to be reviewed is used as a sample label to obtain the training result.

[0009] input the content to be audited into a preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model, wherein the preset text violation detection model takes the content to be audited as a training sample, and takes a probability of each kind of text violation included in the content to be audited as a sample label for training;

[0010] input the emotion audit result and the text audit result into a preset multi-modal violation detection model for analysis to obtain a violation audit result of the content to be audited, wherein the preset multi-modal violation detection model takes the emotion audit result and the text audit result of the content to be audited as training samples, and takes a violation result of the content to be audited as a sample label for training;

[0011] determine whether the content to be audited is in violation according to the violation audit result of the content to be audited;

[0012] if it is determined that the content to be audited is in violation, determine and execute an automatic punishment measure for the content to be audited according to a preset automatic punishment condition.

[0013] Preferably, the process of analyzing the content to be audited by the preset tone emotion detection model to obtain an emotion audit result output by the preset tone emotion detection model includes:

[0014] the preset tone emotion detection model analyzes the content to be audited to determine an audio signal of the content to be audited;

[0015] a speech self-supervised model in the preset tone emotion detection model extracts speech implicit features in the audio signal of the content to be audited, and transmits the speech implicit features to a fully connected layer in the preset tone emotion detection model for analysis to obtain a probability of each tone included in the content to be audited as the emotion audit result.

[0016] Preferably, the process of analyzing the content to be audited by the preset text violation detection model to obtain a text audit result output by the preset text violation detection model includes:

[0017] the preset text violation detection model analyzes the content to be audited to determine text information of the content to be audited;

[0018] The text self-supervised model in the preset text violation detection model extracts text implicit features in text information of the content to be audited, and transmits the text implicit features to a full connection layer in the preset text violation detection model for analysis to obtain a probability of each text information violation included in the content to be audited as the text audit result.

[0019] Preferably, the process in which the preset multi-modal violation detection model analyzes the emotion audit result and the text audit result to obtain a violation audit result of the content to be audited comprises:

[0020] The preset multi-modal violation detection model analyzes the emotion audit result to determine voice implicit features corresponding to the content to be audited.

[0021] The preset multi-modal violation detection model analyzes the text audit result to determine text implicit features corresponding to the content to be audited.

[0022] The preset multi-modal violation detection model fuses the voice implicit features and the text implicit features corresponding to the content to be audited, combines the probability of each tone in the emotion audit result and the probability of each text information violation in the text audit result, and determines a violation audit result of the content to be audited.

[0023] Preferably, the method further comprises:

[0024] If it is determined that the content to be audited does not violate, the violation audit result of the content to be audited is input into a preset natural language processing model for audit to obtain a probability of each violation type included in the text information corresponding to the content to be audited, wherein the preset natural language processing model is trained by taking the text information corresponding to the content to be audited as a training sample and taking the probability of each violation type included in the training text information corresponding to the content to be audited as a sample label.

[0025] If it is determined that no punishment measure is performed on the content to be audited, it is judged whether the probability of each violation type included in the text information corresponding to the content to be audited is greater than a preset threshold.

[0026] If there is a violation type in the text information corresponding to the content to be audited, the probability of which is greater than the preset threshold, it is determined that the violation type of the text information corresponding to the content to be audited is the violation type greater than the preset threshold.

[0027] Preferably, the method further comprises:

[0028] The rule type of the text information corresponding to the content to be audited is greater than the preset threshold, and the rule type is sent to an auditor for confirmation to determine the final audit result of the content to be audited.

[0029] According to the final audit result of the content to be audited, it is judged whether the content to be audited meets the preset automatic penalty condition.

[0030] If the final audit result of the content to be audited is determined to meet the preset automatic penalty condition, the automatic penalty measure for the content to be audited is determined and executed.

[0031] A content audit automatic penalty device comprises:

[0032] A content acquisition unit is configured to acquire the content to be audited.

[0033] A first audit unit is configured to input the content to be audited into a preset emotional tone detection model for analysis to obtain an emotional audit result output by the preset emotional tone detection model, wherein the preset emotional tone detection model is trained by using the content to be audited as a training sample, and the probability of each emotional tone included in the training content to be audited is used as a sample label for training.

[0034] A second audit unit is configured to input the content to be audited into a preset text rule detection model for analysis to obtain a text audit result output by the preset text rule detection model, wherein the preset text rule detection model is trained by using the content to be audited as a training sample, and the probability of each text rule included in the training content to be audited is used as a sample label for training.

[0035] A third audit unit is configured to input the emotional audit result and the text audit result into a preset multi-modal rule detection model for analysis to obtain a rule audit result of the content to be audited, wherein the preset multi-modal rule detection model is trained by using the emotional audit result and the text audit result of the content to be audited as a training sample, and the rule result of the training content to be audited is used as a sample label for training.

[0036] A first judgment unit is configured to judge whether the content to be audited is in violation according to the rule audit result of the content to be audited.

[0037] An automatic penalty execution unit is configured to, when the execution result of the first judgment unit is that the content to be audited is in violation, determine and execute an automatic penalty measure for the content to be audited according to a preset automatic penalty condition.

[0038] Preferably, the first auditing unit comprises:

[0039] an audio signal determination unit configured to determine an audio signal of the content to be audited by using the preset tone emotion detection model;

[0040] a first analysis unit configured to extract speech implicit features in the audio signal of the content to be audited by using a speech self-supervised model in the preset tone emotion detection model, and transmit the speech implicit features to a full connection layer in the preset tone emotion detection model for analysis, so as to obtain a probability of each tone included in the content to be audited as the emotion auditing result.

[0041] A content auditing and automatic punishment device, comprising one or more processors and a memory.

[0042] The memory stores computer readable instructions, and the computer readable instructions are executed by the one or more processors to implement the steps of the content auditing and automatic punishment method according to any one of the preceding descriptions.

[0043] A readable storage medium, the readable storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors to make the one or more processors implement the steps of the content auditing and automatic punishment method according to any one of the preceding descriptions.

[0044] From the above technical solutions, when it is necessary to audit whether the voice content is in violation of the rules, the method provided by the embodiment of the application can obtain the content to be audited, and can input the content to be audited into a preset emotional tone detection model for analysis to obtain an emotional tone audit result output by the preset emotional tone detection model, wherein the preset emotional tone detection model takes the content to be audited as a training sample, and is trained to obtain each probability of a tone included in the content to be audited as a sample label. The audit by the emotional tone detection model can take the tone emotion in the content to be audited as a factor for violation judgment, and can improve the audit accuracy of the deep learning model for the content to be audited. At the same time, the content to be audited can also be input into a preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model, wherein the preset text violation detection model takes the content to be audited as a training sample, and is trained to obtain each probability of a text violation included in the content to be audited as a sample label. The audit by the text violation detection model can take each probability of a text violation included in the content to be audited in the content to be audited as a factor for violation judgment, and can improve the audit accuracy of the deep learning model for the content to be audited. After obtaining the emotional tone audit result and the text audit result, the emotional tone audit result and the text audit result can be further input into a preset multi-modal violation detection model for analysis, so that a violation audit result of the content to be audited can be obtained, wherein the preset multi-modal violation detection model takes the emotional tone audit result and the text audit result of the content to be audited as a training sample, and is trained to obtain a violation result of the content to be audited as a sample label. The use of the preset multi-modal violation detection model to fuse and analyze the emotional tone audit result and the text audit result can improve the accuracy of the violation audit result of the content to be audited. Therefore, after obtaining the violation audit result of the content to be audited, whether the content to be audited is in violation of the rules can be determined according to the violation audit result of the content to be audited. If it is determined that the content to be audited is in violation of the rules, an automatic punishment measure for the content to be audited can be determined and executed according to a preset automatic punishment condition, so that the audit efficiency of the content to be audited can be effectively improved.

[0045] The method provided by the embodiment of the application can analyze the tone emotion information in the content to be audited by using a tone emotion analysis model, can effectively improve the accuracy of the audit result of the content to be audited, and can further analyze the tone emotion audit result and the text information audit result of the content to be audited by using a multi-modal rule detection model. After it is determined that the content to be audited contains rule violation content, the automatic punishment measure for the content to be audited can be automatically determined and executed, which can effectively improve the audit accuracy and efficiency of the content to be audited, can effectively reduce the cost of manual audit, and can improve the timeliness of punishment for rule violation behavior. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 A flowchart of a method for implementing content audit automatic punishment provided by the embodiment of the present application;

[0048] Figure 2 A structural schematic diagram of a content audit automatic punishment device for the embodiment of the present application;

[0049] Figure 3 A hardware structure block diagram of a content audit automatic punishment device disclosed by the embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] With the gradual tightening of supervision on the field of pan-entertainment socialization or live broadcast at the present stage, in order to comply with the regulatory requirements and reduce the risk of violation, audio and video live broadcast companies need to monitor the content generated by their users and take measures such as banning users for rule violation and illegal information.

[0052] However, as the number of users of the audio-video live broadcast platform increases, the number of audio-video generated daily also increases. According to incomplete statistics, some audio-video live broadcast platforms can generate tens of thousands to hundreds of thousands of hours of audio-video content per day. If only artificial monitoring is used to monitor whether the large amount of audio-video content is in violation, the human cost required is huge and boring.

[0053] With the development of deep learning, people have gradually begun to use deep learning models to cooperate with artificial auditing to audit audio-video, use models to determine information that is in violation, and then manually review the information that is in violation before taking corresponding punishment on the audio-video that is in violation.

[0054] In actual application, the audio-video content auditing model generally outputs a violation score. When the violation score of a certain audio-video content is greater than a preset threshold, the audio-video content auditing model considers that the information is in violation.

[0055] In actual application, the audio-video content auditing model generally defines two constraint conditions of recall rate and accuracy rate to help audit whether the audio-video content is in violation.

[0056] Among them, the recall rate and accuracy rate defined by the audio-video content auditing model can be mutually restricted, and the recall rate and accuracy rate defined by the audio-video content auditing model can be adjusted by adjusting the preset threshold.

[0057] For example, the comparison table of the results of the auditing model predicting violation and non-violation can be defined as follows:

[0058] Table 1 Comparison table of results of auditing model predicting violation and non-violation

[0059] Audit model predicts violation Audit model predicts no violation True violation True Positive (TP) False Negative (FN) True no violation False Positive (FP) True Negative (TN)

[0060] Among them,

[0061] The recall rate Recall = TP / (TP+FN) can be set

[0062] The accuracy rate Precision = TP / (TP+FP) can be set

[0063] For example,

[0064] The lower the preset threshold is set, the more the number of contents in violation in the audio-video to be audited that the audio-video content auditing model considers. Assuming that the audio-video content auditing model considers all samples to be in violation, the number of samples that the audio-video content auditing model predicts to be not in violation is 0, and the recall rate is 100%, but the accuracy rate of the audio-video content auditing result of the audio-video content auditing model is very low.

[0065] Conversely, the preset threshold is increased, and the number of real violation samples and the number of real non-violation samples in the audio and video content review model are reduced, wherein the number of real violation samples in the audio and video content review model is greater than the number of real non-violation samples when the high violation score, and the accuracy of the audio and video content review model on the review result of the audio and video content is likely to increase, but the recall rate is relatively low.

[0066] In some content review scenarios in actual application process, the recall rate is more valued because the missed violation information is more serious than the model misjudgment as violation information, the former affects platform safety, and the latter only wastes a little manpower.

[0067] When some business values the recall rate, the number of samples that need to be reviewed by manual is more, and the problem of manpower shortage may occur, therefore, in order to cover all the violation samples predicted by the review model by manual, more people may be needed to review, which may lead to the increase of labor cost; the increase of samples to be reviewed and the limited manpower resource make the punishment response slow, which may lead to the violation behavior not being stopped in time.

[0068] In addition, the current general review process is that the audio first passes through the speech-to-text module (ASR), and then the text information in the speech enters the natural language processing (NLP) module to determine whether the audio and video content is in violation. However, the current general review process loses the tone information in the speech, and cannot determine the emotional state of the speaker, which can feedback whether the content of the speech is in violation. The loss of the tone information in the speech affects the accuracy of the entire review result.

[0069] In view of the fact that most of the current content review automatic punishment schemes are difficult to adapt to complex and changeable business needs, the applicant has researched a content review automatic punishment scheme, which can use a tone emotion analysis model to analyze the tone emotion information in the content to be reviewed, effectively improve the accuracy of the review result of the content to be reviewed, further use a multi-modal violation detection model to fuse and analyze the tone emotion review result and the text information review result of the content to be reviewed, and after determining that the content to be reviewed contains violation content, automatically determine and execute the automatic punishment measures for the content to be reviewed, which can effectively improve the review accuracy and efficiency of the content to be reviewed, and also effectively reduce the cost of manual review and improve the timeliness of punishment for violation behavior.

[0070] The method provided by the embodiments of the present application can be used in many general or special-purpose computing device environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments that include any of the above devices or devices, and the like.

[0071] The embodiment of the present application provides a content review automatic punishment method, which can be applied to various audio and video publishing system audio and video review systems, and can also be applied to various computer terminals or intelligent terminals, and the execution subject can be a processor or a server of a computer terminal or an intelligent terminal.

[0072] The content review automatic punishment method provided by the embodiment of the present application will be described below Figure 1 , and the flow of the content review automatic punishment method provided by the embodiment of the present application is shown in Figure 1 , which can include the following steps:

[0073] Step S101, obtaining the content to be reviewed.

[0074] Specifically, in the actual application process, when the audio and video content generated by the user needs to be reviewed for violation, the content to be reviewed can be obtained first, so that the content to be reviewed can be analyzed to determine whether it is in violation.

[0075] The content to be reviewed can include audio content, video content or video data containing audio content.

[0076] Step S102, inputting the content to be reviewed into a preset emotional detection model for analysis to obtain an emotional review result output by the preset emotional detection model.

[0077] Specifically, after determining the content to be reviewed, in order to analyze the emotional information included in the content to be reviewed, the content to be reviewed can be input into the preset emotional detection model for analysis, so that the emotional review result output by the preset emotional detection model can be obtained.

[0078] The preset emotional detection model can be trained by taking the content to be reviewed as a training sample, and taking the probability of each tone included in the training content to be reviewed as a sample label.

[0079] The emotional review result can include the emotional information included in the voice data in the content to be reviewed.

[0080] The emotional information included in the voice data in the content to be reviewed can reflect the emotion included in the speech of the speaker.

[0081] The basic emotions can include happiness, anger, fear and sadness, and the four emotions are the most basic and original emotions.

[0082]

[0083] ​They are associated with basic needs and often have high levels of tension.

[0084] Happiness is the psychological pleasure and comfort that an individual experiences when tension is released after achieving a desired goal. The intensity of happiness is related to the ease and probability of achieving the goal. The more difficult it is to achieve a goal, the more intense the experience of happiness when it is achieved. For example, people feel very happy when they solve a difficult geometry problem after a great deal of effort, but they do not experience such happiness when they solve an easy problem. In addition, people experience greater happiness when their wishes are fulfilled at unexpected times and in unexpected places.

[0085] Anger is an emotional experience that results from the accumulation of tension when a desire is not fulfilled and the behavior to achieve the desire is repeatedly blocked. When a desire is blocked, it is called frustration. When an individual understands the cause of the frustration, he or she usually reacts with anger toward the person or thing that caused the frustration. If an individual does not understand the cause of the frustration, he or she usually only feels disappointed, not angry. Explicit anger often induces aggressive behavior.

[0086] Fear is an emotional experience that an individual experiences when he or she attempts to escape or avoid a situation. This experience is caused by a lack of ability to deal with the frightening situation. For example, when people encounter a strong natural disaster such as an earthquake, they are often terrified because they are unable to cope with the situation. Children often have more fear experiences because of their lack of experience and ability, such as fear of the dark and fear of small animals. As people grow older, they learn more ways to deal with problems, and some things that used to cause fear no longer frighten them. However, when familiar environments change unexpectedly, such as a fire or a large-scale power outage, fear is again triggered. Fear is highly contagious. When one person is afraid, others are often affected and become afraid and uneasy.

[0087] Sorrow is an emotional experience that an individual experiences when he or she loses something that he or she values and pursues. Failure and separation can cause sorrow. The intensity of sorrow depends on the psychological value of the lost thing to the subject. The greater the psychological value, the more intense the sorrow. The death of a close relative causes extreme sorrow, which is different from the sorrow caused by the loss of a general friend. Sorrow can be divided into regret, disappointment, sadness, and grief according to intensity.

[0088] Therefore, by analyzing the emotional information of the tone of the content to be audited, it can be determined whether the speaker has an angry, angry, sarcastic, or abusive behavior towards others.

[0089] Step S103, inputting the content to be audited into a preset text violation detection model for analysis, to obtain a text audit result output by the preset text violation detection model.

[0090] Specifically, after determining the content to be audited, in order to analyze the text information included in the content to be audited, the content to be audited can be input into the preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model.

[0091] The preset text violation detection model can be trained by taking the content to be audited as a training sample, and the probability of each type of text violation included in the training content to be audited as a sample label.

[0092] The text audit result can include text information included in the content to be audited.

[0093] The text audit result can include text information included in the content to be audited.

[0094] The text information included in the content to be audited can feedback whether the text content corresponding to the words spoken by the speaker includes text content that violates regulations.

[0095] For example,

[0096] The text information included in the content to be audited can feedback whether the text information included in the content to be audited includes text information that insults or satirizes people.

[0097] Step S104, inputting the emotion audit result and the text audit result into a preset multi-modal violation detection model for analysis to obtain a violation audit result of the content to be audited.

[0098] Specifically, as described above, the present application can analyze the speech tone emotion information corresponding to the speech included in the content to be audited and the audit result of the text information included in the content to be audited.

[0099] Therefore, by analyzing the first emotion audit information and the first text audit information, it can be determined whether the content to be audited includes information that violates regulations, so as to determine whether the content to be audited needs to be punished for violation.

[0100] Further, in order to improve the accuracy of the audit of the content to be audited, after determining the speech tone emotion information corresponding to the speech included in the content to be audited and the audit result of the text information included in the content to be audited, the emotion audit result and the text audit result can be input into the preset multi-modal violation detection model for analysis, so as to obtain a violation audit result of the content to be audited.

[0101] The preset multi-modal violation detection model can take the emotion review result and the text review result of the content to be reviewed as training samples, take the violation result of the content to be reviewed as a sample label, and be trained.

[0102] The preset multi-modal violation detection model can perform fusion analysis on the emotion review result and the text review result, consider the violation of the tone emotion of the content to be reviewed, and also consider the violation of the text information in the content to be reviewed, thereby effectively improving the accuracy of the violation review of the content to be reviewed.

[0103] In step S105, whether the content to be reviewed is in violation is determined according to the violation review result of the content to be reviewed.

[0104] Specifically, as known from the above description, the multi-modal violation detection model can be used to determine the violation review result of the content to be reviewed.

[0105] In actual application, after obtaining the violation review result of the content to be reviewed, whether the content to be reviewed is in violation can be further determined.

[0106] If it is determined that the content to be reviewed is in violation, it is indicated that the content to be reviewed needs to be punished, and step S106 can be performed.

[0107] In step S106, an automatic punishment measure for the content to be reviewed is determined and performed according to a preset automatic punishment condition.

[0108] Specifically, as known from the above description, whether the content to be reviewed is in violation can be determined, and if it is determined that the content to be reviewed is in violation, an automatic punishment measure for the content to be reviewed can be determined and performed according to a preset automatic punishment condition.

[0109] The preset automatic punishment condition can be set according to the supervision requirements of a relevant supervision department on a violation behavior.

[0110] For example, the automatic punishment condition can be set as that when the content to be reviewed includes a behavior of insulting others, the content to be reviewed can be given a punishment of deletion or muting.

[0111] From the above technical solutions, the method provided by the embodiment of the present application can analyze the tone emotion information in the content to be audited by using a tone emotion analysis model, which can effectively improve the accuracy of the audit result of the content to be audited. At the same time, by using a text violation detection model to analyze the text information corresponding to the content to be audited, the tone emotion audit result and the text information audit result of the content to be audited can be further analyzed by using a multi-modal violation detection model. After determining that the content to be audited contains illegal content, automatic punishment measures for the content to be audited can be automatically determined and executed, which can effectively improve the accuracy and efficiency of the audit of the content to be audited, and can effectively reduce the cost of manual audit and improve the timeliness of punishment for illegal behavior.

[0112] As can be seen from the above introduction, the first emotion audit information of the content to be audited can be obtained by analyzing the content to be audited. Next, the process is introduced, which can include the following steps:

[0113] In step S201, the preset tone emotion detection model analyzes the content to be audited to determine the audio signal of the content to be audited.

[0114] Specifically, in actual application, the content to be audited can include audio data.

[0115] The audio data can be embodied by an audio signal.

[0116] Therefore, the content to be audited can include an audio signal.

[0117] The audio signal included in the content to be audited can reflect the speech characteristics of the content to be audited. The speech characteristics of the content to be audited can feedback the tone emotion information of the speech corresponding to the content to be audited. Understanding the tone emotion information of the speech corresponding to the content to be audited can better understand the emotional state of the content to be audited.

[0118] The preset tone emotion detection model can analyze the content to be audited to determine the audio signal of the content to be audited, so that whether the content to be audited contains illegal emotional state information can be determined by analyzing the audio signal of the content to be audited.

[0119] In step S202, the speech self-supervised model in the preset tone emotion detection model extracts the speech implicit features in the audio signal of the content to be audited, and transmits the speech implicit features to the full connection layer in the preset tone emotion detection model for analysis, to obtain the probability of each tone included in the content to be audited as the emotion audit result.

[0120] Specifically, in actual application, the preset emotional tone detection model can include a speech self-supervised model and a full connection layer. After determining the audio signal, the speech self-supervised model in the preset emotional tone detection model can extract speech implicit features in the audio signal of the content to be audited, and can transmit the speech implicit features to the full connection layer in the preset emotional tone detection model for analysis, so as to obtain the probability of each emotional tone included in the content to be audited as the emotional audit result.

[0121] For example,

[0122] By analyzing the audio signal, the probabilities of happiness, anger, fear and sadness emotions included in the speech corresponding to the content to be audited can be determined.

[0123] As can be seen from the above technical solutions, the method provided by the embodiments of the present application can analyze the content to be audited to obtain the first emotional audit information, which helps to understand whether the content to be audited includes behaviors of insulting, satirizing or laughing at others by analyzing the first emotional audit information. This helps to improve the accuracy of the rule violation audit of the content to be audited.

[0124] As can be seen from the above introduction, the present application can analyze the content to be audited to obtain the first text audit information. Next, the process will be introduced, which can include the following steps:

[0125] Step S301, the preset text rule detection model analyzes the content to be audited to determine the text information of the content to be audited.

[0126] Specifically, in actual application, the content to be audited can include text information.

[0127] Among them,

[0128] The text information included in the content to be audited can reflect the text features of the content to be audited. The text features of the content to be audited can feed back the text content information corresponding to the content to be audited. Understanding the text content information corresponding to the content to be audited can better understand whether the content to be audited has rule violation content.

[0129] The preset text rule detection model analyzes the content to be audited to determine the text information of the content to be audited, so that whether the content to be audited includes rule violation text content can be determined by analyzing the text information of the content to be audited.

[0130] In step S302, the text self-supervised model in the preset text violation detection model extracts text implicit features in the text information of the content to be audited, and transmits the text implicit features to the full connection layer in the preset text violation detection model for analysis, to obtain a probability of each text information violation included in the content to be audited as the text audit result.

[0131] Specifically, in actual application, the preset text violation detection model can include a text self-supervised model and a full connection layer. After determining the text information, the text self-supervised model in the preset text violation detection model can extract text implicit features in the text information of the content to be audited, and can transmit the text implicit features to the full connection layer in the preset text violation detection model for analysis, so as to obtain a probability of each text information violation included in the content to be audited as the text audit result.

[0132] For example,

[0133] By analyzing the text information, the probability of the content to be audited corresponding to the expression of happy, angry, fearful, and sad emotions corresponding to the text information can be determined.

[0134] As can be seen from the above technical solutions, the method provided by the embodiments of the present application can analyze the content to be audited to obtain the first text audit information, which helps to understand whether the content to be audited includes behaviors of insulting, satirizing, or laughing at others by analyzing the first text audit information, and helps to improve the accuracy of violation audit of the content to be audited.

[0135] As can be seen from the above introduction, the present application can analyze the first emotion audit information and the first text audit information to obtain the violation audit result of the content to be audited. Next, the process is introduced, which can include the following steps:

[0136] In step S401, the preset multi-modal violation detection model analyzes the emotion audit result to determine the voice implicit features corresponding to the content to be audited.

[0137] Specifically, as can be seen from the above introduction, the method provided by the embodiments of the present application can determine the emotion audit result corresponding to the content to be audited.

[0138] The emotion audit result includes voice implicit features corresponding to the content to be audited.

[0139] After determining the emotion audit result, if the emotion audit result is directly used to analyze whether the content to be audited is in violation, the influence of the text information corresponding to the audited content is easily ignored, and the accuracy is not high.

[0140] Therefore, after determining the emotion review result, the preset multi-modal violation detection model can analyze the emotion review result to determine the voice implicit feature corresponding to the content to be reviewed, so that whether the content to be reviewed violates the rules can be analyzed in combination with the text information corresponding to the reviewed content.

[0141] In step S402, the preset multi-modal violation detection model analyzes the text review result to determine the text implicit feature corresponding to the content to be reviewed.

[0142] Specifically, as introduced above, the method provided in the embodiments of the present application can determine the text review result corresponding to the content to be reviewed.

[0143] The text review result includes the text implicit feature corresponding to the content to be reviewed.

[0144] After determining the text review result, if the text review result is directly used to analyze whether the content to be reviewed violates the rules, the influence of the voice feature of the content to be reviewed is easily neglected, and the accuracy is not high.

[0145] Therefore, after determining the text review result, the preset multi-modal violation detection model can analyze the text review result to determine the text implicit feature corresponding to the content to be reviewed, so that whether the content to be reviewed violates the rules can be analyzed in combination with the emotion information corresponding to the reviewed content.

[0146] In step S403, the preset multi-modal violation detection model fuses the voice implicit feature and the text implicit feature corresponding to the content to be reviewed, combines the probability of each tone in the emotion review result and the probability of each text information violating the rules in the text review result, and determines the violation review result of the content to be reviewed.

[0147] Specifically, as introduced above, the emotion review result and the text review result both reflect the violation situation of the content to be reviewed.

[0148] Therefore, after determining the voice implicit feature and the text implicit feature corresponding to the content to be reviewed, the preset multi-modal violation detection model can fuse the voice implicit feature and the text implicit feature corresponding to the content to be reviewed, combine the probability of each tone in the emotion review result and the probability of each text information violating the rules in the text review result, and perform violation analysis on the content to be reviewed, so as to determine the violation review result of the content to be reviewed.

[0149] For example,

[0150] The fusion analysis of the output of the multi-modal violation detection model can be as follows:

[0151] P r = αP j + βP rb + γP rw

[0152] P s = αP h + βP sb + γP sw

[0153] Wherein,

[0154] P r may represent the probability of abusive behavior;

[0155] P j may represent the probability of excited emotion;

[0156] P rb may represent the probability of abusive text;

[0157] P rw may represent the probability of abusive violation;

[0158] P s may represent the probability of sexual behavior;

[0159] P h may represent the probability of sexual tone;

[0160] P sb may represent the probability of sexual text;

[0161] P sw may represent the probability of sexual violation;

[0162] In addition,

[0163] P j / P h may be the output probability of the tone emotion detection model;

[0164] P rb / P sb may be the output probability of the text violation detection model;

[0165] P rw / P sw may be the output probability of the multi-modal violation detection model.

[0166] α, β, γ are respective weights, which can be learned by linear regression; or can be artificially adjusted.

[0167] From the above technical solutions, the method provided by the embodiment of the application can fuse the voice implicit features and the text implicit features of the content to be audited, and combine the probability of each tone in the emotion audit result and the probability of each text information violation in the text audit result to perform violation analysis on the content to be audited, so as to determine the violation audit result of the content to be audited, so as to determine whether the content to be audited needs to be automatically punished.

[0168] In actual application, when the multi-modal detection model fails to detect, the application can use the natural language processing model to re-audit the content to be audited to verify whether the audit result of the content to be audited by the multi-modal detection model is accurate. Next, the process is introduced, which can include the following steps:

[0169] Step S501, if it is determined that the content to be audited does not violate the rules, the violation audit result of the content to be audited is input into a preset natural language processing model for auditing to obtain the probability of each type of violation included in the text information corresponding to the content to be audited.

[0170] Specifically, in actual application, the multi-modal violation detection model may fail to detect or audit, so that the behavior that should be punished cannot be punished in time, causing adverse effects.

[0171] Therefore, if it is determined that the content to be audited does not violate the rules, the violation audit result of the content to be audited can be input into the preset natural language processing model for auditing, so that the probability of each type of violation included in the text information corresponding to the content to be audited can be obtained.

[0172] The preset natural language processing model can be trained by taking the text information corresponding to the content to be audited as a training sample and taking the probability of each type of violation included in the text information corresponding to the content to be audited as a sample label.

[0173] Step S502, if it is determined that the punishment measure for the content to be audited is not executed, it is judged whether the probability of each type of violation included in the text information corresponding to the content to be audited is greater than a preset threshold.

[0174] Specifically, as known from the above description, in actual application, when the multi-modal violation detection model determines that the content to be audited has a violation behavior, the content to be audited can be automatically punished.

[0175] In actual application, when it is determined that the punishment measure is not performed on the content to be audited, it is further determined whether each rule type probability included in the text information corresponding to the content to be audited is greater than a preset threshold, so as to determine whether the content to be audited needs to be punished for rule violation.

[0176] For example,

[0177] If there is a rule type in the text information corresponding to the content to be audited, the probability of which is greater than the preset threshold, it is indicated that the content to be audited needs to be punished for rule violation.

[0178] In step S503, it is determined that the rule type of the text information corresponding to the content to be audited is a rule type greater than the preset threshold.

[0179] Specifically, if there is a rule type in the text information corresponding to the content to be audited, the probability of which is greater than the preset threshold, it is indicated that there is a rule type that needs to be punished for rule violation. Therefore, it can be determined that the rule type of the text information corresponding to the content to be audited is a rule type greater than the preset threshold, so that the content to be audited can be punished for rule violation according to the rule type of the content to be audited.

[0180] As can be seen from the above technical solutions, when the multi-modal detection model misses detection, the method provided by the embodiments of the present application can use the natural language processing model to re-audit the content to be audited to verify whether the audit result of the multi-modal detection model on the content to be audited is accurate, and further determine the rule type of the content to be audited, so that the content to be audited can be punished for rule violation according to the rule type of the content to be audited.

[0181] In actual application, when the multi-modal detection model misses detection, the natural language processing model can be used to audit the content to be audited, and then artificial auditing is performed to verify whether the audit result of the multi-modal detection model on the content to be audited is accurate. Next, the process is introduced, which can include the following steps:

[0182] In step S601, the rule type of the text information corresponding to the content to be audited is greater than the preset threshold, and the rule type is sent to an auditing personnel for confirmation to determine the final audit result of the content to be audited.

[0183] Specifically, as introduced above, when the multi-modal rule detection model misses detection or it is determined that the content to be audited does not have rule violation, the method provided by the embodiments of the present application can use the natural language processing model to audit the content to be audited.

[0184] To determine the accuracy of the violation review of the content to be reviewed, the method provided by the embodiment of the application can further send the violation type of the text information corresponding to the content to be reviewed that is greater than the preset threshold to the reviewer for confirmation to determine the final review result of the content to be reviewed after the content to be reviewed is reviewed by using the natural language processing model, so as to ensure the accuracy of the violation review result of the content to be reviewed.

[0185] Step S602, judging whether the content to be reviewed meets the preset automatic penalty condition according to the final review result of the content to be reviewed.

[0186] Specifically, in actual application, after determining the final review result of the content to be reviewed, not all types of review results corresponding to the content to be reviewed need to be penalized.

[0187] Therefore, after determining the final review result of the content to be reviewed, whether the content to be reviewed meets the preset automatic penalty condition can be judged according to the final review result of the content to be reviewed, so that the penalty measures for the content to be reviewed can be determined according to the preset automatic penalty condition.

[0188] If it is determined that the content to be reviewed meets the preset automatic penalty condition according to the final review result of the content to be reviewed, it means that the content to be reviewed needs to be automatically penalized, and then step S603 can be performed.

[0189] Step S603, determining and performing the automatic penalty measures for the content to be reviewed.

[0190] Specifically, in actual application, if it is determined that the content to be reviewed meets the preset automatic penalty condition according to the final review result of the content to be reviewed, it means that the content to be reviewed needs to be automatically penalized, and then the automatic penalty measures for the content to be reviewed can be determined and performed.

[0191] The automatic penalty measures can be set according to the violation type of the content to be reviewed.

[0192] The automatic penalty measures can be set according to the violation type of the content to be reviewed.

[0193] The necessary condition for performing automatic penalty can be set according to the accuracy of the multi-modal violation detection model and the natural language processing model.

[0194] For example, the necessary condition for performing automatic punishment can be set as that the accuracy of the multi-modal violation detection model and the natural language processing model reaches 99% or above, and various violation type corresponding punishment measures are performed, otherwise, normal speech is easily misjudged as a violation and the user is punished, which will cause the user of the application to lose.

[0195] In actual application process, when some business needs to consider recall rate, the accuracy of the multi-modal violation detection model and the natural language processing model can be set to fluctuate between 20% and 70%, at this time, the necessary condition for performing automatic punishment can be set as that the accuracy of the multi-modal violation detection model and the natural language processing model reaches 20% to 70%, and various violation type corresponding punishment measures are performed.

[0196] From the above technical solution, it can be seen that when the multi-modal detection model misses detection, the natural language processing model can be used to audit the content to be audited, and then artificial auditing is performed to verify whether the audit result of the multi-modal detection model on the content to be audited is accurate. In order to determine whether the content to be audited needs to be punished automatically according to the final audit result of the content to be audited.

[0197] The content auditing automatic punishment device provided by the embodiment of the application is described below, and the content auditing automatic punishment device described below can be referred to in conjunction with the content auditing automatic punishment method described above.

[0198] Referring to Figure 2 , Figure 2 A structural schematic diagram of a content auditing automatic punishment device disclosed by the embodiment of the application.

[0199] As Figure 2 shown, the content auditing automatic punishment device can include:

[0200] The content acquisition unit 101 is configured to acquire the content to be audited.

[0201] The first auditing unit 102 is configured to input the content to be audited into a preset tone emotion detection model for analysis to obtain an emotion audit result output by the preset tone emotion detection model, wherein the preset tone emotion detection model is trained by using the content to be audited as a training sample, and each type of tone included in the training content to be audited is used as a sample label for training.

[0202] The second auditing unit 103 is configured to input the content to be audited into a preset text violation detection model for analysis to obtain a text auditing result output by the preset text violation detection model, wherein the preset text violation detection model is trained by using the content to be audited as a training sample, and a probability of each type of text violation included in the content to be audited is used as a sample label for training.

[0203] The third auditing unit 104 is configured to input the emotional auditing result and the text auditing result into a preset multi-modal violation detection model for analysis to obtain a violation auditing result of the content to be audited, wherein the preset multi-modal violation detection model is trained by using the emotional auditing result and the text auditing result of the content to be audited as a training sample, and a violation result of the content to be audited is used as a sample label for training.

[0204] The first judging unit 105 is configured to judge whether the content to be audited is in violation according to the violation auditing result of the content to be audited.

[0205] The first automatic punishment execution unit 106 is configured to, when the execution result of the first judging unit is that the content to be audited is in violation, determine and execute an automatic punishment measure on the content to be audited according to a preset automatic punishment condition.

[0206] From the above technical solutions, when it is necessary to audit whether the voice content is in violation of the rules, the device provided by the embodiment of the application can obtain the content to be audited, and can input the content to be audited into a preset emotional tone detection model for analysis to obtain an emotional tone audit result output by the preset emotional tone detection model, wherein the preset emotional tone detection model takes the content to be audited as a training sample, and takes the probability of each emotional tone included in the content to be audited as a sample label for training. The audit by the emotional tone detection model can take the emotional tone in the content to be audited as a factor for violation judgment, and can improve the audit accuracy of the deep learning model for the content to be audited. At the same time, the content to be audited can also be input into a preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model, wherein the preset text violation detection model takes the content to be audited as a training sample, and takes the probability of each text violation included in the content to be audited as a sample label for training. The audit by the text violation detection model can take the probability of each text violation included in the content to be audited in the content to be audited as a factor for violation judgment, and can improve the audit accuracy of the deep learning model for the content to be audited. After obtaining the emotional tone audit result and the text audit result, the emotional tone audit result and the text audit result can be further input into a preset multi-modal violation detection model for analysis, so that the violation audit result of the content to be audited can be obtained, wherein the preset multi-modal violation detection model takes the emotional tone audit result and the text audit result of the content to be audited as a training sample, and takes the violation result of the content to be audited as a sample label for training. The use of the preset multi-modal violation detection model to fuse and analyze the emotional tone audit result and the text audit result can improve the accuracy of the violation audit result of the content to be audited. Therefore, after obtaining the violation audit result of the content to be audited, whether the content to be audited is in violation of the rules can be determined according to the violation audit result of the content to be audited. If it is determined that the content to be audited is in violation of the rules, automatic punishment measures for the content to be audited can be determined and executed according to preset automatic punishment conditions, so that the audit efficiency of the content to be audited can be effectively improved.

[0207] The device provided by the embodiments of the present application can analyze the tone emotion information in the content to be audited by using a tone emotion analysis model, effectively improve the accuracy of the audit result of the content to be audited, and further analyze the tone emotion audit result and the text information audit result of the content to be audited by using a multi-modal violation detection model. After determining that the content to be audited contains illegal content, the automatic punishment measure for the content to be audited can be automatically determined and executed, which can effectively improve the audit accuracy and efficiency of the content to be audited, effectively reduce the cost of manual audit, and improve the timeliness of punishment for illegal behavior.

[0208] Further optionally, the first audit unit 102 can comprise:

[0209] An audio signal determination unit configured to determine an audio signal of the content to be audited by using the preset tone emotion detection model.

[0210] A first analysis unit configured to extract speech implicit features in the audio signal of the content to be audited by using a speech self-supervised model in the preset tone emotion detection model, and transmit the speech implicit features to a full connection layer in the preset tone emotion detection model for analysis, so as to obtain the probability of each tone included in the content to be audited as the emotion audit result.

[0211] Further optionally, the second audit unit 103 can comprise:

[0212] A text information determination unit configured to determine text information of the content to be audited by using the preset text violation detection model.

[0213] A second analysis unit configured to extract text implicit features in the text information of the content to be audited by using a text self-supervised model in the preset text violation detection model, and transmit the text implicit features to a full connection layer in the preset text violation detection model for analysis, so as to obtain the probability of each text information violation included in the content to be audited as the text audit result.

[0214] Further optionally, the third audit unit 104 can comprise:

[0215] A speech feature determination unit configured to determine speech implicit features corresponding to the content to be audited by using the preset multi-modal violation detection model.

[0216] The text feature determination unit is configured to analyze the text review result by using the preset multi-modal violation detection model, and determine the text implicit feature corresponding to the content to be reviewed.

[0217] The third analysis unit is configured to fuse the voice implicit feature and the text implicit feature corresponding to the content to be reviewed by using the preset multi-modal violation detection model, combine the probability of each tone in the emotion review result and the probability of each text information violation in the text review result, and determine the violation review result of the content to be reviewed.

[0218] Further, the device can further include:

[0219] The fourth review unit is configured to input the violation review result of the content to be reviewed into a preset natural language processing model for review when the execution result of the first judgment unit 105 is that the content to be reviewed does not violate, to obtain the probability of each violation type included in the text information corresponding to the content to be reviewed, wherein the preset natural language processing model is trained by using the text information corresponding to the content to be reviewed as a training sample, and the probability of each violation type included in the training of the text information corresponding to the content to be reviewed as a sample label, and is obtained by training.

[0220] The second judgment unit is configured to determine whether the probability of each violation type included in the text information corresponding to the content to be reviewed is greater than a preset threshold when it is determined that the punishment measure for the content to be reviewed is not executed.

[0221] The violation type determination unit is configured to determine that the violation type of the text information corresponding to the content to be reviewed is the violation type greater than the preset threshold when the execution result of the second judgment unit is that the probability of the violation type existing in the text information corresponding to the content to be reviewed is greater than the preset threshold.

[0222] Further, the device can further include:

[0223] The review result confirmation unit is configured to send the violation type of the text information corresponding to the content to be reviewed, which is greater than the preset threshold, to a reviewer for confirmation, to determine the final review result of the content to be reviewed.

[0224] The third judgment unit is configured to determine whether the content to be reviewed satisfies a preset automatic punishment condition according to the final review result of the content to be reviewed.

[0225] The second automatic punishment execution unit is configured to, when the execution result of the third judgment unit determines that the final audit result of the content to be audited is correct, determine that the content to be audited meets the preset condition of automatic punishment, and then determine and execute an automatic punishment measure on the content to be audited.

[0226] The specific processing procedure of each unit included in the content audit automatic punishment apparatus can refer to the related description in the foregoing content audit automatic punishment method, and will not be described here again.

[0227] The content audit automatic punishment apparatus provided in the embodiments of the present application can be applied to a content audit automatic punishment device, such as a terminal, for example, a mobile phone, a computer, and the like. Optionally, Figure 3 A hardware structure block diagram of the content audit automatic punishment device is shown, which can refer to Figure 3 The hardware structure of the content audit automatic punishment device can include at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0228] In the embodiments of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 complete communication with each other through the communication bus 4.

[0229] The processor 1 can be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application, etc.

[0230] The memory 3 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0231] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement each processing procedure in the foregoing terminal content audit automatic punishment scheme.

[0232] The embodiments of the present application also provide a readable storage medium, which can store a program suitable for processor execution, and the program is used to implement each processing procedure in the foregoing terminal content audit automatic punishment scheme.

[0233] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are not intended to denote the presence of any such actual relationship or order. Moreover, the terms "include", "have", or any other variant thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles, or apparatuses that comprise a list of elements are not required to comprise only those elements on the list, but can include other elements not expressly listed or inherent to such processes, methods, articles, or apparatuses.

[0234] The various embodiments in the specification are described with progression in this order of description. Embodiments of each order of description can be combined with embodiments of the other orders of description.

[0235] The above description of disclosed embodiments provides enabling concepts for those skilled in the art to carry out or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. The various embodiments can be combined with each other. Therefore, the present application will not be limited to the embodiments shown herein but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A content review automatic penalty method, characterized by, The method comprises the following steps: obtaining content to be audited; inputting the content to be audited into a preset emotional tone detection model for analysis to obtain an emotional audit result output by the preset emotional tone detection model, wherein the preset emotional tone detection model is trained by taking the content to be audited as a training sample, and the probability of each tone included in the training of the content to be audited is taken as a sample label for training; inputting the content to be audited into a preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model, wherein the preset text violation detection model is trained by taking the content to be audited as a training sample, and the probability of each text violation included in the training of the content to be audited is taken as a sample label for training; inputting the emotional audit result and the text audit result into a preset multi-modal violation detection model for analysis to obtain a violation audit result of the content to be audited, wherein the preset multi-modal violation detection model is trained by taking the emotional audit result and the text audit result of the content to be audited as training samples, and the violation result of the training of the content to be audited is taken as a sample label for training; the process of obtaining the violation audit result by the preset multi-modal violation detection model comprises: analyzing the emotional audit result to determine the voice implicit features corresponding to the content to be audited; analyzing the text audit result to determine the text implicit features corresponding to the content to be audited; fusing the voice implicit features and the text implicit features corresponding to the content to be audited, combining the probability of each tone in the emotional audit result and the probability of each text information violation in the text audit result to determine the violation audit result of the content to be audited; judging whether the content to be audited is in violation according to the violation audit result of the content to be audited; if it is determined that the content to be audited is in violation, determining and executing an automatic punishment measure for the content to be audited according to a preset automatic punishment condition.

2. The method of claim 1, wherein, The process of analyzing the content to be audited by the preset emotional tone detection model to obtain the emotional audit result output by the preset emotional tone detection model comprises: the preset emotional tone detection model analyzes the content to be audited to determine the audio signal of the content to be audited; a voice self-supervised model in the preset emotional tone detection model extracts voice implicit features in the audio signal of the content to be audited, and transmits the voice implicit features to a fully connected layer in the preset emotional tone detection model for analysis to obtain the probability of each tone included in the content to be audited as the emotional audit result.

3. The method of claim 1, wherein, The process of analyzing the content to be audited by the preset text violation detection model to obtain the text audit result output by the preset text violation detection model comprises: the preset text violation detection model analyzes the content to be audited to determine the text information of the content to be audited; The text self-supervised model in the preset text violation detection model extracts a text implicit feature in text information of the content to be audited, and transmits the text implicit feature to a full connection layer in the preset text violation detection model for analysis to obtain a probability of each text information violation included in the content to be audited as the text audit result.

4. The method of claim 1, wherein, The method further includes: If it is determined that the content to be audited is not in violation, a violation audit result of the content to be audited is input into a preset natural language processing model for auditing to obtain a probability of each violation type included in text information corresponding to the content to be audited, wherein the preset natural language processing model is trained by taking the text information corresponding to the content to be audited as a training sample and taking the probability of each violation type included in the text information corresponding to the content to be audited as a sample label. If it is determined that no punishment measure is performed on the content to be audited, it is judged whether the probability of each violation type included in the text information corresponding to the content to be audited is greater than a preset threshold value. If there is a violation type in the text information corresponding to the content to be audited, the probability of which is greater than the preset threshold value, it is determined that the violation type of the text information corresponding to the content to be audited is the violation type greater than the preset threshold value.

5. The method of claim 4, wherein, The method further includes: The violation type of the text information corresponding to the content to be audited, which is greater than the preset threshold value, is sent to an auditing personnel for confirmation to determine a final audit result of the content to be audited. According to the final audit result of the content to be audited, it is judged whether the content to be audited satisfies a preset automatic punishment condition. If it is determined, according to the final audit result of the content to be audited, that the content to be audited satisfies the preset automatic punishment condition, an automatic punishment measure is determined and performed on the content to be audited.

6. A content review automatic penalty apparatus characterized by comprising: It includes: A content acquisition unit configured to acquire content to be audited; A first audit unit configured to input the content to be audited into a preset tone emotion detection model for analysis to obtain an emotion audit result output by the preset tone emotion detection model, wherein the preset tone emotion detection model is trained by taking the content to be audited as a training sample and taking a probability of each tone included in the content to be audited as a sample label; A second audit unit configured to input the content to be audited into a preset text violation detection model for analysis to obtain a text audit result output by the preset text violation detection model, wherein the preset text violation detection model is trained by taking the content to be audited as a training sample and taking a probability of each text violation included in the content to be audited as a sample label. a third auditing unit, configured to input the emotion auditing result and the text auditing result into a preset multi-modal rule violation detection model for analysis to obtain a rule violation auditing result of the content to be audited, wherein the preset multi-modal rule violation detection model is trained by taking the emotion auditing result and the text auditing result of the content to be audited as training samples and taking a rule violation result of the content to be audited as a sample label, and the third auditing unit includes: a voice feature determination unit, configured to determine voice implicit features corresponding to the content to be audited by the preset multi-modal rule violation detection model analyzing the emotion auditing result; a text feature determination unit, configured to determine text implicit features corresponding to the content to be audited by the preset multi-modal rule violation detection model analyzing the text auditing result; and a third analysis unit, configured to determine the rule violation auditing result of the content to be audited by the preset multi-modal rule violation detection model fusing the voice implicit features and the text implicit features corresponding to the content to be audited, combining a probability of each tone in the emotion auditing result and a probability of each text information rule violation in the text auditing result; a first judgment unit, configured to judge whether the content to be audited is in violation of rules according to the rule violation auditing result of the content to be audited; an automatic punishment execution unit, configured to determine and execute an automatic punishment measure for the content to be audited according to a preset automatic punishment condition when the execution result of the first judgment unit is that the content to be audited is in violation of rules.

7. The apparatus of claim 6, wherein, The first auditing unit includes: an audio signal determination unit, configured to determine an audio signal of the content to be audited by the preset tone emotion detection model analyzing the content to be audited; a first analysis unit, configured to extract voice implicit features in the audio signal of the content to be audited by a voice self-supervised model in the preset tone emotion detection model, and transmit the voice implicit features to a full connection layer in the preset tone emotion detection model for analysis to obtain a probability of each tone included in the content to be audited as the emotion auditing result.

8. A content review automatic penalty apparatus characterized by comprising: One or more processors and a memory; The memory stores computer readable instructions, and the computer readable instructions are executed by the one or more processors to implement the steps of the content auditing and automatic punishment method in any one of claims 1 to 5. The readable storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors to cause the one or more processors to implement the steps of the content auditing and automatic punishment method in any one of claims 1 to 5.

9. A readable storage medium characterized by: ​

Images