Audio violation identification method and device, storage medium and electronic terminal

Abstract

The invention provides an audio violation recognition method and device, a storage medium and an electronic terminal, and can obtain a to-be-recognized audio and an audio content text corresponding to the to-be-recognized audio, and input the audio content text into a text violation recognition model to obtain a text recognition result. And if the text recognition result reflects that violation words exist in the audio content text, obtaining volume information of the to-be-recognized audio, and obtaining a violation recognition result of the to-be-recognized audio according to the volume information. Therefore, rough classification can be performed by using the text violation identification model and the audio content text, and deep analysis is performed in combination with the volume information of the to-be-identified audio under the condition that the text violation is determined, so as to finally determine whether the to-be-identified audio is violated or not. According to the method and the device, the audio is subjected to multi-mode violation recognition by combining the data of the audio and text modes, so that the false recognition rate in the audio violation recognition process can be effectively reduced, and the accuracy of audio violation recognition can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, storage medium and electronic terminal for identifying audio violations. Background Art

[0002] With the widespread adoption of real-time voice social platforms, their application has expanded from real-time multi-person voice communication to a variety of scenarios, including live streaming and voice communities. Due to the frequent user interactions and complex voice scenarios of these platforms, these platforms place more stringent requirements on audio content violation identification technology. However, the inventors have discovered that existing technologies struggle to effectively distinguish between highly offensive insults and less aggressive catchphrases when identifying audio violations, and struggle to accurately distinguish between gaming jargon and illegal audio content. This results in a high rate of false positives, which in turn affects the accuracy of audio violation identification. Summary of the Invention

[0003] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies, especially the technical defect of low accuracy in violation identification in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a method for identifying audio violations, comprising:

[0005] Obtaining the audio to be recognized and the audio content text corresponding to the audio to be recognized;

[0006] Inputting the audio content text into a pre-trained text violation recognition model to obtain a text recognition result;

[0007] If the text recognition result reflects that the audio content text contains illegal words, volume information of the audio to be recognized is obtained, and an illegal recognition result of the audio to be recognized is obtained according to the volume information.

[0008] In one embodiment, obtaining volume information of the audio to be identified and obtaining a violation identification result of the audio to be identified based on the volume information includes:

[0009] Divide the audio to be recognized into N audio segments, and obtain the segment volume of each audio segment and the overall volume of the audio to be recognized; wherein N is a positive integer;

[0010] If the overall volume is less than or equal to the preset volume threshold, it is determined that the audio to be identified does not violate the rules;

[0011] If the overall volume is greater than the preset volume threshold, the violation identification result is obtained according to the volume of each of the segments.

[0012] In one embodiment, obtaining the violation identification result based on the volume of each of the clips includes:

[0013] Extracting a text violation score and at least one identified violation word from the text recognition result; wherein the identified violation word is a violation word that appears in the audio content text;

[0014] determining a volume violation score based on the volume of each of the segments and the at least one identified violation word;

[0015] Performing a weighted summation on the volume violation score and the text violation score to obtain a comprehensive violation score;

[0016] The violation identification result is determined based on the comprehensive violation score.

[0017] In one embodiment, performing a weighted summation of the volume violation score and the text violation score to obtain a comprehensive violation score includes:

[0018] Determining a volume weight coefficient and a text weight coefficient based on the total number of identified illegal words; wherein the volume weight coefficient is positively correlated with the total number of identified illegal words, and the text weight coefficient is negatively correlated with the total number of identified illegal words;

[0019] The volume violation score and the text violation score are weighted and summed according to the volume weight coefficient and the text weight coefficient to obtain the comprehensive violation score.

[0020] In one embodiment, determining the volume weight coefficient and the text weight coefficient respectively based on the total number of identified illegal words includes:

[0021] Determining an incremental coefficient according to the overall volume; wherein the incremental coefficient is positively correlated with the overall volume;

[0022] Calculating the product of the increment coefficient and the total number of the recognized illegal words to obtain the volume weight coefficient;

[0023] The difference between the preset value and the volume weight coefficient is used as the text weight coefficient.

[0024] In one embodiment, determining a volume violation score based on the volume of each segment and the at least one identified violation word comprises:

[0025] Determining a timestamp at which a target violation word appears in the audio to be recognized; wherein the target violation word is any one of the at least one recognition violation word;

[0026] Determining, in each of the audio segments, a target audio segment corresponding to the target illegal word according to a timestamp corresponding to the target illegal word;

[0027] The volume of the segment corresponding to the target audio segment is used as the volume of the word corresponding to the target illegal word;

[0028] Using a preset volume normalization threshold, the word volume is normalized to obtain a word volume score for the target illegal word;

[0029] The volume violation score is obtained according to the word volume score of the at least one identified violation word.

[0030] In one embodiment, determining the violation identification result based on the comprehensive violation score includes:

[0031] matching the comprehensive violation score with a plurality of preset violation classification intervals;

[0032] Whether the audio to be identified violates the rules is determined based on the matching result, and the violation level of the audio to be identified is determined.

[0033] In a second aspect, an embodiment of the present application provides an audio violation identification device, comprising:

[0034] An acquisition module is used to acquire the audio to be recognized and the audio content text corresponding to the audio to be recognized;

[0035] A text violation recognition module is used to input the audio content text into a pre-trained text violation recognition model and obtain a text recognition result;

[0036] The violation identification result acquisition module is used to obtain the volume information of the audio to be identified if the text recognition result shows that the audio content text contains a violation word, and obtain the violation identification result of the audio to be identified based on the volume information.

[0037] In a third aspect, an embodiment of the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the audio violation identification method described in any of the above embodiments.

[0038] In a fourth aspect, an embodiment of the present application provides an electronic terminal, the electronic terminal comprising: one or more processors, and a memory;

[0039] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the audio violation identification method described in any of the above embodiments are performed.

[0040] In the audio violation identification method, device, storage medium and electronic terminal provided in some embodiments of the present application, the audio to be identified and the audio content text corresponding to the audio to be identified can be obtained, and the audio content text is input into the text violation identification model to obtain a text recognition result. If the text recognition result reflects that there are violation words in the audio content text, the volume information of the audio to be identified is obtained, and the violation identification result of the audio to be identified is obtained based on the volume information. In this way, the text violation identification model and the audio content text can be used for coarse classification, and when it is determined that the text is in violation, an in-depth analysis is performed in combination with the volume information of the audio to be identified to ultimately determine whether the audio to be identified is in violation. The present application performs multimodal violation identification on audio by combining data from two modes, audio and text, thereby effectively reducing the false recognition rate in the audio violation identification process, and thus improving the accuracy of audio violation identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 A flowchart of a method for identifying audio violations in some embodiments;

[0043] Figure 2 is a schematic structural diagram of an audio violation identification device in some embodiments;

[0044] Figure 3 This is a diagram of the internal structure of an electronic terminal in some embodiments. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0046] In some embodiments, the present application provides a method for identifying audio violations. The following embodiments illustrate the method using an electronic terminal as an example. It is understood that the electronic terminal described in this application can be a terminal device with data processing capabilities, including but not limited to various personal computers, notebook computers, laptop computers, tablet computers, smartphones, Internet of Things devices, and portable wearable devices. Internet of Things devices can include smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices can include smart watches, smart bracelets, head-mounted devices, etc.

[0047] like Figure 1 As shown, the method provided in this application may specifically include the following steps:

[0048] S102: Acquire the audio to be recognized and the audio content text corresponding to the audio to be recognized.

[0049] It is understood that the audio to be identified refers to the audio that needs to be identified for violations. Its audio length, audio format, audio acquisition method, etc. can be determined according to actual circumstances, and this application does not impose specific restrictions on this. Audio content text refers to the text content corresponding to the audio to be identified, which can display the voice content of the audio to be identified in text form.

[0050] In some examples, in order to facilitate subsequent processing, improve the efficiency of violation identification and reduce the hardware requirements of the electronic terminal, the present application can obtain a real-time audio stream and segment the real-time audio stream according to a preset duration, thereby obtaining one or more segments of audio to be identified. If multiple segments of audio to be identified are obtained, for each segment of audio to be identified, the electronic terminal can perform violation identification on the audio to be identified according to the process described in steps S104 to S106. Among them, the preset duration can be determined according to the actual situation, for example, it can be 5 seconds, 10 seconds or 30 seconds, and this application does not impose specific restrictions on this.

[0051] It should be noted that the present application can obtain the audio content text in any manner, and the present application does not impose any specific restrictions on this. It only requires that the audio content text can record the voice content of the audio to be recognized in text form.

[0052] In some examples, the electronic terminal may use ASR (Automatic Speech Recognition) technology to perform speech recognition on the audio to be recognized, thereby obtaining the speech content text. For example, the electronic terminal may input the audio to be recognized into a pre-built ASR model and obtain the audio content text output by the ASR model. The ASR model may be a deep learning model, its encoder may be a shared encoder, and its decoder may include a CTC (Connectionist Temporal Classification) decoder and an Attention decoder. After the audio to be recognized is input into the ASR model, the shared encoder extracts audio features from the audio to be recognized and inputs the audio features into the CTC decoder and the Attention decoder, respectively. The CTC decoder may perform decoding operations based on the audio features and input several candidate texts obtained by decoding into the Attention decoder. The Attention decoder may rescore the several candidate texts based on the audio features to obtain the audio content text.

[0053] S104: Input the audio content text into a pre-trained text violation recognition model to obtain a text recognition result.

[0054] The text violation identification model refers to a model used to identify violations in audio content text, which can be obtained through pre-training. It is understood that the text violation identification model can be obtained through any training method and can be implemented based on a model of any structure and any principle. For example, the text violation identification model can be implemented using models such as BERT (Bidirectional Encoder Representation from Transformers), textCNN, and fasttext, and this application does not impose specific restrictions on this.

[0055] In this step, the electronic terminal can input the audio content text into the text violation recognition model to use the text violation recognition model to identify whether the audio content text contains violation words and perform violation recognition and rough violation classification of the text. After the audio content text is input into the text violation recognition model, a text recognition result output by the text violation recognition model can be obtained.

[0056] It is understood that the text violation identification result is used to reflect whether the audio content text contains any illegal words and whether there is a text violation problem. For example, if the text violation result includes at least one illegal word, it indicates that the audio content text contains illegal words, and it can be determined that there is a text violation problem. Conversely, if the text violation result does not include any illegal words, it indicates that the audio content text does not contain any illegal words.

[0057] S106: If the text recognition result indicates that the audio content text contains illegal words, volume information of the audio to be recognized is obtained, and an illegal recognition result of the audio to be recognized is obtained based on the volume information.

[0058] In this step, if at least one illegal word is identified in the audio content, an in-depth analysis can be performed based on the volume information of the audio to be identified, ultimately determining whether the audio to be identified violates the rules. Because volume information can reflect information such as the speaker's tone, attitude, and emotions, this application can use the volume information to determine whether the illegal word in the audio content corresponds to a highly offensive insult or a less offensive or non-offensive normal expression, and thus obtain a violation identification result.

[0059] In this application, a text violation recognition model and audio content text can be used for rough classification. If a text violation is determined, the volume information of the audio to be recognized can be combined for in-depth analysis to ultimately determine whether the audio to be recognized violates the law. This application combines audio and text data to perform multimodal violation recognition on audio, thereby effectively reducing the false recognition rate in the audio violation recognition process and thus improving the accuracy of audio violation recognition.

[0060] In some embodiments, the present application further includes the following step: if the text recognition result indicates that the audio content text does not contain any illegal words, then the audio to be identified can be directly determined to be non-illegal. In this embodiment, if the audio content text does not include any illegal words, step S106 can be skipped and the audio to be identified can be directly determined to be non-illegal. In this way, resource consumption can be reduced and the efficiency of audio violation identification can be improved.

[0061] In some embodiments, obtaining volume information of the audio to be identified and obtaining a violation identification result of the audio to be identified based on the volume information includes:

[0062] Step A1: Segment the audio to be recognized into N audio segments, and obtain the segment volume of each audio segment and the overall volume of the audio to be recognized; where N is a positive integer;

[0063] Step A3: If the overall volume is less than or equal to the preset volume threshold, it is determined that the audio to be identified does not violate the rules;

[0064] Step A5: If the overall volume is greater than the preset volume threshold, a violation identification result is obtained based on the volume of each segment.

[0065] In this embodiment, during the audio modality violation identification process, the electronic terminal may segment the audio to be identified into N audio segments to facilitate volume statistics. It is understood that N can be a preset positive integer or dynamically adjusted based on actual circumstances. This application does not impose any specific restrictions on the method for obtaining N or the audio segmentation method.

[0066] In some examples, the present application can adaptively segment the audio to be identified based on the volume continuity of the audio to be identified, thereby dividing the audio to be identified into N audio segments. Specifically, first, the volume amplitude of the audio to be identified within a range near each sampling point can be calculated separately according to random sampling or average sampling. For each sampling point, if the volume amplitude near the sampling point is within the preset volume amplitude range, and the duration is greater than the preset duration threshold, then the sampling point and the audio near the sampling point can be divided into the same segment. If the volume amplitude near the sampling point is not within the preset volume amplitude range, or the duration is less than or equal to the preset duration threshold, then the range can be divided into smaller segments, and the jump point can be found as the segmentation point. According to the above steps, the segmentation point and the audio near it are judged for consistency until all audio segments are determined.

[0067] After audio segmentation, the present application can determine the segment volume of each audio segment and obtain N segment volumes. In addition, the present application can determine the overall volume of the audio to be recognized. It can be understood that the segment volume of the audio segment is used to reflect the volume information of the audio segment, and the overall volume is used to reflect the volume information of the audio to be recognized as a whole.

[0068] It should be noted that this application may use any method to obtain the segment volume and overall volume, and this application does not impose any specific restrictions on this. In some examples, this application may use the average of N segment volumes as the overall volume of the audio to be recognized. For example, when N=5, and the segment volumes of the audio segments are 3470, 3981, 2142, 3169, and 2866 respectively, the overall volume may be 3125.

[0069] If the overall volume of the audio being identified is less than or equal to a preset threshold, the offending words in the audio content can be determined to correspond to game commentary terms or mild, non-offensive catchphrases, and therefore the audio being identified is considered acceptable. If the overall volume exceeds the threshold, further analysis and judgment can be performed based on the volume of N segments.

[0070] In this embodiment, during the audio modality violation identification process, the electronic device can make a preliminary violation determination based on the overall volume of the audio being identified. If the overall volume is less than or equal to a preset volume threshold, the audio being identified can be directly determined to be non-violation, without the need to analyze the volume of individual segments. This reduces resource consumption and improves the efficiency of audio violation identification.

[0071] In some embodiments, obtaining violation identification results based on the volume of each clip includes:

[0072] Step A51: extracting a text violation score and at least one identified violation word from the text recognition result; wherein the identified violation word is a violation word that appears in the audio content text;

[0073] Step A53: determining a volume violation score based on the volume of each segment and at least one identified violation word;

[0074] Step A55: performing a weighted summation of the volume violation score and the text violation score to obtain a comprehensive violation score;

[0075] Step A57: Determine a violation identification result based on the comprehensive violation score.

[0076] In this embodiment, the text recognition result may include a text violation score and identified violation words included in the audio content text. The text violation score may be a probability of audio violation based on the text modality, and the identified violation words refer to each violation word that appears in the audio content text.

[0077] If the overall volume of the audio to be recognized exceeds a preset volume threshold, all recognized violation words can be extracted from the text recognition results. A volume violation score can be determined based on the volume of the N segments and all recognized violation words. The volume violation score can be a probability of audio violation based on the audio modality.

[0078] The electronic terminal can perform a weighted summation of the volume violation score and the text violation score, thereby combining the audio violation probabilities of the text and audio modalities to obtain a comprehensive violation score. Based on this comprehensive violation score, the terminal can determine whether the audio being identified violates a violation. This can further reduce the false positive rate of audio violation recognition, thereby further improving recognition accuracy.

[0079] It is understood that during the weighted sum calculation process, the volume weight coefficient corresponding to the volume violation score and the text weight coefficient corresponding to the text violation score can be determined in any manner. For example, the volume weight coefficient and the text weight coefficient can be predetermined or dynamically adjusted based on actual conditions.

[0080] In some embodiments, a weighted sum of the volume violation score and the text violation score is performed to obtain a comprehensive violation score, including:

[0081] Step A551: Determine a volume weight coefficient and a text weight coefficient based on the total number of recognized illegal words; wherein the volume weight coefficient is positively correlated with the total number of recognized illegal words, and the text weight coefficient is negatively correlated with the total number of recognized illegal words;

[0082] Step A553: Based on the volume weight coefficient and the text weight coefficient, a weighted sum is performed on the volume violation score and the text violation score to obtain a comprehensive violation score.

[0083] Specifically, the number of illegal words in an audio content can, to a certain extent, reflect information such as semantics, tone, and attitude. The more illegal words appear in an audio content, the stronger the tone. Conversely, the fewer illegal words appear in an audio content, the less strong the tone.

[0084] In this embodiment, the electronic terminal can determine the volume weight coefficient and the text weight coefficient respectively according to the total number of recognized illegal words included in the audio content text. The greater the total number of recognized illegal words and the stronger the tone, the greater the proportion of tone in audio violation identification should be, and the greater the volume weight coefficient. Conversely, the fewer the total number of recognized illegal words and the weaker the tone, the proportion of tone in audio violation identification should be reduced, and the proportion of text modality in audio violation identification should be increased, and the greater the text weight coefficient.

[0085] For example, the volume weight coefficient weight1 = 1 - n_keyword * delta_weight, and the text weight coefficient weight2 = n_keyword × delta_weight. Here, n_keyword is the total number of identified illegal words, and delta_weight is a pre-set incremental coefficient. Furthermore, weight1 and weight2 can have upper and lower limits, with the lower limit being 0 and the upper limit being 1.

[0086] In this embodiment, the tone of the audio to be identified can be determined based on the total number of identified illegal words, and the proportion of volume in illegal identification can be determined dynamically based on the tone, thereby effectively balancing the illegal probabilities of the audio modality and the text modality, further reducing the misrecognition rate and improving accuracy.

[0087] In some embodiments, determining a volume weight coefficient and a text weight coefficient based on the total number of identified illegal words includes:

[0088] Step A5511: determining an incremental coefficient based on the overall volume; wherein the incremental coefficient is positively correlated with the overall volume;

[0089] Step A5513: Calculate the product of the incremental coefficient and the total number of recognized illegal words, and obtain the volume weight coefficient;

[0090] Step A5517: The difference between the preset value and the volume weight coefficient is used as the text weight coefficient.

[0091] In this embodiment, the delta_weight increment factor can be determined based on the overall volume, allowing the delta_weight increment factor to dynamically change based on the overall volume. This allows the tone of the audio being recognized to be determined based on the overall volume and the total number of recognized violation words, further balancing the violation probabilities of the audio and text modalities, thereby reducing the false recognition rate and improving accuracy.

[0092] Specifically, the higher the overall volume, the stronger the tone. In this case, the incremental coefficient can be adjusted upward to increase the proportion of the volume weight coefficient in violation identification. Conversely, the lower the overall volume, the weaker the tone. In this case, the incremental coefficient can be adjusted downward to reduce the proportion of the volume weight coefficient in violation identification and increase the proportion of the text modality in violation identification.

[0093] It should be noted that the present application can implement step A5511 in any manner, and the present application does not impose any specific restrictions on this. It only requires that the incremental coefficient is positively correlated with the overall volume. For example, the present application can pre-establish a corresponding relationship between the overall volume value range and the incremental coefficient. For example, if the overall volume is between 0 and 1000, the incremental coefficient is 0.01; if the overall volume is between 1000 and 2000, the incremental coefficient is 0.05; if the overall volume is between 2000 and 3000, the incremental coefficient is 0.1; if the overall volume is between 3000 and 4000, the incremental coefficient is 0.15; if the overall volume is between 4000 and 5000, the incremental coefficient is 0.25; if the overall volume is between 5000 and 6000, the incremental coefficient is 0.35. When determining the gain coefficient, the electronic terminal can first determine the value range into which the overall volume falls, and determine the gain coefficient accordingly. For example, when the overall volume is 500, the gain coefficient can be determined to be 0.01.

[0094] In some embodiments, determining a volume violation score based on the volume of each segment and at least one identified violation word includes:

[0095] Step A531: Determine a timestamp of a target offending word appearing in the audio to be recognized; wherein the target offending word is any one of the at least one recognized offending word;

[0096] Step A533: determining a target audio segment corresponding to the target offending word in each audio segment according to the timestamp corresponding to the target offending word;

[0097] Step A535: using the volume of the segment corresponding to the target audio segment as the volume of the word corresponding to the target illegal word;

[0098] Step A537: Using a preset volume normalization threshold, normalize the word volume to obtain a word volume score for the target offending word;

[0099] Step A539: Obtain a volume violation score based on the word volume score of at least one identified violation word.

[0100] In this embodiment, in the process of calculating the volume violation score, the timestamp of each recognized violation word appearing in the audio to be recognized can be determined respectively, and the audio segment corresponding to the recognized violation word can be determined based on the timestamp, thereby obtaining the word volume corresponding to the recognized violation word.

[0101] For example, if a recognized offending word appears in the first audio segment based on its timestamp, the segment volume of the first audio segment can be used as the word volume corresponding to the recognized offending word. For another example, if a recognized offending word appears in both the first and second audio segments based on its timestamp, the average of the segment volumes of the first and second audio segments can be used as the word volume corresponding to the recognized offending word.

[0102] It is understood that the value of N may affect the accuracy of the word volume. The larger N is, the more accurate the word volume is. Therefore, the present application may determine the value of N according to the accuracy requirement of the word volume, for example, N may be 20.

[0103] When the word volume corresponding to each identified offending word is obtained, the present application can perform normalization calculations on each word volume separately through a preset volume normalization threshold, and obtain the word volume score corresponding to the identified offending word. For example, when the audio content text includes a first identified offending word and a second identified offending word, the word volume of the first identified offending word is 2113, the word volume of the second identified offending word is 381, and the volume normalization threshold is 6000, then after normalization calculation, the word volume score of the first identified offending word is 2113 / 6000=0.35, and the word volume score of the second identified offending word is 381 / 6000=0.06.

[0104] After obtaining the word volume scores corresponding to each identified violation word, the present application can derive the volume violation score of the audio to be recognized based on the total word volume scores. For example, the present application can use the sum of all word volume scores as the volume violation score. That is, in the above example, the volume violation score is 0.35 + 0.06 = 0.41.

[0105] In this embodiment, by identifying the timestamp of the offending word in the audio to be identified, the audio segment corresponding to the identified offending word is determined, and the volume of the corresponding word is obtained accordingly. In this way, the intensity of the identified offending word can be accurately determined, thereby improving the accuracy of subsequent violation identification.

[0106] In some embodiments, determining a violation identification result based on the comprehensive violation score includes:

[0107] Step A571: matching the comprehensive violation score with a plurality of preset violation classification intervals;

[0108] Step A573: Determine whether the audio to be identified violates the rules based on the matching results, and determine the violation level of the audio to be identified.

[0109] In this embodiment, the electronic terminal can match the comprehensive violation score with multiple preset violation classification intervals, determine the target classification interval within which the comprehensive violation score falls, and then determine whether the identified audio is in violation based on the target classification interval. If the identified audio is in violation, the violation level can be determined based on the target classification interval. In this way, the specific violation level can be determined based on volume information, thereby increasing the flexibility of operational violation handling and providing more detailed basis for operations.

[0110] It is understood that the number of preset violation classification intervals can be determined based on actual circumstances, for example, 3, 5, 10, etc., and this application does not impose specific limitations on this. In some examples, the number of preset violation classification intervals can be 3, with the first preset violation classification interval being 0-0.3, the second preset violation classification interval being 0.3-0.7, and the third preset violation classification interval being 0.7-1. When the comprehensive violation score is 0.45, it can be determined that the audio to be identified is in violation and the violation level is moderate.

[0111] The following describes an audio violation identification device provided in an embodiment of the present application. The audio violation identification device described below and the audio violation identification method described above can be referenced to each other.

[0112] In some embodiments, as Figure 2 As shown, the present application provides an audio violation identification device 200, comprising:

[0113] An acquisition module 202 is configured to acquire audio to be recognized and audio content text corresponding to the audio to be recognized;

[0114] A text violation identification module 204 is configured to input the audio content text into a pre-trained text violation identification model and obtain a text recognition result;

[0115] The violation identification result acquisition module 206 is configured to obtain volume information of the audio to be identified if the text recognition result indicates that the audio content text contains a violation word, and obtain a violation identification result of the audio to be identified based on the volume information.

[0116] In some embodiments, the violation identification result acquisition module 206 of the present application includes:

[0117] an audio segmentation unit, configured to segment the audio to be recognized into N audio segments, and obtain the segment volume of each audio segment and the overall volume of the audio to be recognized; wherein N is a positive integer;

[0118] a first violation judgment unit, configured to determine that the audio to be identified does not violate the rules if the overall volume is less than or equal to a preset volume threshold;

[0119] The second violation judgment unit is configured to obtain the violation identification result according to the volume of each of the segments if the overall volume is greater than the preset volume threshold.

[0120] In some embodiments, the second violation determination unit of the present application includes:

[0121] A word extraction unit, configured to extract a text violation score and at least one identified violation word from the text recognition result; wherein the identified violation word is a violation word that appears in the audio content text;

[0122] a volume violation score determining unit, configured to determine a volume violation score based on the volume of each of the segments and the at least one identified violation word;

[0123] a comprehensive violation score determination unit, configured to perform a weighted summation of the volume violation score and the text violation score to obtain a comprehensive violation score;

[0124] A violation identification result obtaining unit is configured to determine the violation identification result based on the comprehensive violation score.

[0125] In some embodiments, the comprehensive violation score determination unit of the present application includes:

[0126] a weight coefficient determination unit, configured to determine a volume weight coefficient and a text weight coefficient according to the total number of identified illegal words; wherein the volume weight coefficient is positively correlated with the total number of identified illegal words, and the text weight coefficient is negatively correlated with the total number of identified illegal words;

[0127] A weighted summing unit is configured to perform weighted summation on the volume violation score and the text violation score according to the volume weight coefficient and the text weight coefficient to obtain the comprehensive violation score.

[0128] In some embodiments, the weight coefficient determination unit of the present application includes:

[0129] an incremental coefficient determining unit, configured to determine an incremental coefficient according to the overall volume; wherein the incremental coefficient is positively correlated with the overall volume;

[0130] a volume coefficient calculation unit, configured to calculate the product of the incremental coefficient and the total number of the identified illegal words, and obtain the volume weight coefficient;

[0131] The text coefficient calculation unit is configured to use the difference between a preset value and the volume weight coefficient as the text weight coefficient.

[0132] In some embodiments, the volume violation score determination unit of the present application includes:

[0133] a timestamp determination unit, configured to determine a timestamp at which a target violation word appears in the audio to be recognized; wherein the target violation word is any one of the at least one recognized violation word;

[0134] a target audio segment determining unit, configured to determine a target audio segment corresponding to the target violating word in each of the audio segments according to a timestamp corresponding to the target violating word;

[0135] a word volume acquisition unit, configured to use the segment volume corresponding to the target audio segment as the word volume corresponding to the target illegal word;

[0136] a normalization unit, configured to perform normalization calculation on the word volume using a preset volume normalization threshold to obtain a word volume score for the target illegal word;

[0137] The violation score calculation unit is configured to obtain the volume violation score according to the word volume score of the at least one identified violation word.

[0138] In some embodiments, the violation identification result acquisition unit of the present application includes:

[0139] a matching unit, configured to match the comprehensive violation score with a plurality of preset violation classification intervals;

[0140] The level determination unit is used to determine whether the audio to be identified violates the rules based on the matching result, and determine the violation level of the audio to be identified.

[0141] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the audio violation identification method in any embodiment.

[0142] In one embodiment, the present application also provides an electronic terminal having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the audio violation identification method in any embodiment.

[0143] Schematically, Figure 3 This is a schematic diagram of the internal structure of an electronic terminal provided in an embodiment of the present application. Figure 3 The electronic terminal 900 includes a processing component 902, which further includes one or more processors, and a memory resource represented by a memory 901 for storing instructions executable by the processing component 902, such as an application. The application stored in the memory 901 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 902 is configured to execute the instructions to perform the steps of the audio violation identification method described in any of the above embodiments.

[0144] The electronic terminal 900 may further include a power supply component 903 configured to perform power management of the electronic terminal 900, a wired or wireless network interface 904 configured to connect the electronic terminal 900 to a network, and an input / output (I / O) interface 905. The electronic terminal 900 may operate based on an operating system stored in the memory 901, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0145] Those skilled in the art will understand that the internal structure of the electronic terminal shown in the present application is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic terminal to which the scheme of the present application is applied. The specific electronic terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0146] Finally, it should be noted that, in this article, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further restriction, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element. Herein, "one," "said," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. A plurality refers to at least two, such as 2, 3, 5, or 8. "And / or" includes any and all combinations of the relevant listed items.

[0147] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0148] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying audio violations, characterized in that: include: Obtaining the audio to be recognized and the audio content text corresponding to the audio to be recognized; Inputting the audio content text into a pre-trained text violation recognition model to obtain a text recognition result; If the text recognition result reflects that the audio content text contains illegal words, volume information of the audio to be recognized is obtained, and an illegal recognition result of the audio to be recognized is obtained according to the volume information.

2. The method according to claim 1, characterized in that The acquiring volume information of the audio to be identified and obtaining a violation identification result of the audio to be identified according to the volume information includes: Divide the audio to be recognized into N audio segments, and obtain the segment volume of each audio segment and the overall volume of the audio to be recognized; wherein N is a positive integer; If the overall volume is less than or equal to the preset volume threshold, it is determined that the audio to be identified does not violate the rules; If the overall volume is greater than the preset volume threshold, the violation identification result is obtained according to the volume of each of the segments.

3. The method according to claim 2, characterized in that Obtaining the violation identification result according to the volume of each segment includes: Extracting a text violation score and at least one identified violation word from the text recognition result; wherein the identified violation word is a violation word that appears in the audio content text; determining a volume violation score based on the volume of each of the segments and the at least one identified violation word; Performing a weighted summation on the volume violation score and the text violation score to obtain a comprehensive violation score; The violation identification result is determined based on the comprehensive violation score.

4. The method according to claim 3, characterized in that The weighted summing of the volume violation score and the text violation score to obtain a comprehensive violation score includes: Determining a volume weight coefficient and a text weight coefficient based on the total number of identified illegal words; wherein the volume weight coefficient is positively correlated with the total number of identified illegal words, and the text weight coefficient is negatively correlated with the total number of identified illegal words; The volume violation score and the text violation score are weighted and summed according to the volume weight coefficient and the text weight coefficient to obtain the comprehensive violation score.

5. The method according to claim 4, characterized in that Determining the volume weight coefficient and the text weight coefficient based on the total number of identified illegal words includes: Determining an incremental coefficient according to the overall volume; wherein the incremental coefficient is positively correlated with the overall volume; Calculating the product of the increment coefficient and the total number of the recognized illegal words to obtain the volume weight coefficient; The difference between the preset value and the volume weight coefficient is used as the text weight coefficient.

6. The method according to any one of claims 3 to 5, characterized in that The step of determining a volume violation score according to the volume of each segment and the at least one identified violation word comprises: Determining a timestamp at which a target violation word appears in the audio to be recognized; wherein the target violation word is any one of the at least one recognition violation word; Determining, in each of the audio segments, a target audio segment corresponding to the target illegal word according to a timestamp corresponding to the target illegal word; The volume of the segment corresponding to the target audio segment is used as the volume of the word corresponding to the target illegal word; Using a preset volume normalization threshold, the word volume is normalized to obtain a word volume score for the target illegal word; The volume violation score is obtained according to the word volume score of the at least one identified violation word.

7. The method according to any one of claims 3 to 5, characterized in that Determining the violation identification result based on the comprehensive violation score includes: matching the comprehensive violation score with a plurality of preset violation classification intervals; Whether the audio to be identified violates the rules is determined based on the matching result, and the violation level of the audio to be identified is determined.

8. An audio violation identification device, characterized in that: include: An acquisition module is used to acquire the audio to be recognized and the audio content text corresponding to the audio to be recognized; A text violation recognition module is used to input the audio content text into a pre-trained text violation recognition model and obtain a text recognition result; The violation identification result acquisition module is used to obtain the volume information of the audio to be identified if the text recognition result shows that the audio content text contains a violation word, and obtain the violation identification result of the audio to be identified based on the volume information.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the audio violation identification method according to any one of claims 1 to 7.

10. An electronic terminal, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the audio violation identification method according to any one of claims 1 to 7.

Images