Segmentation violation review method and device for audio, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for segmented audio violation review, relating to the field of audio processing. It involves obtaining the ASR text of the audio to be processed, then splitting the ASR text into multiple sub-texts corresponding to the sub-audio segments of the audio to be processed. Each sub-text is then input into a trained violation review model to obtain a target violation label vector for each sub-text. Finally, based on the target violation label vector of each sub-text, a target violation type label is determined for each sub-audio segment. The target violation type label indicates whether the sub-audio segment is a violation or not. Thus, this automated review method can effectively and promptly identify violation audio, and the segmented review approach of this solution can accurately locate small, violating sub-audio segments within a longer audio file.

Description

Technical Field

[0001] This invention relates to the field of audio processing, and more specifically, to a method, apparatus, electronic device, and storage medium for segmented violation review of audio. Background Technology

[0002] With the continuous development of information technology, audio live streaming services have attracted widespread attention due to their novel format and rich content. However, audio live streaming services may contain a lot of illegal content, therefore, effective supervision of the video and audio content appearing in real time during audio live streams is necessary.

[0003] Taking audio sharing platforms as an example, the current technology is not very effective in supervising the audio albums of audio broadcasters or the audio in live broadcast rooms. One way is to manually check the selected broadcasters' voices or to manually monitor the selected live broadcast room audio. Another way is to send the audio files to the corresponding review platform for review when the audio broadcaster publishes the audio, or to send the audio files generated during the live broadcast to the corresponding review platform for review after the live broadcast ends.

[0004] However, the first regulatory method, due to the randomness of manual spot checks, inevitably misses important violations, making it impossible to punish or ban illegal live streams in a timely manner. The second method, which requires manual listening to a large number of audio files, is labor-intensive and inefficient. Even if the illegal audio is accurately located, due to the lag in this review method, the illegal audio has already spread on the Internet, making it impossible to punish or ban illegal live streams in a timely manner, thus causing adverse social impact. Summary of the Invention

[0005] The purpose of this invention is to provide a method, apparatus, electronic device, and storage medium for segmented violation review of audio, so as to improve the problems existing in the prior art.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides a method for segmented violation review of audio, comprising:

[0008] Acquire the audio to be processed, and perform speech recognition on the audio to be processed to obtain the ASR text to be processed;

[0009] Based on preset splitting rules, the ASR text to be processed is split into multiple sub-texts; each sub-text corresponds to a segment of the audio to be processed.

[0010] Each of the sub-texts is input into the trained violation review model to obtain a target violation label vector for each sub-text. The target violation label vector uses different values ​​to represent the probability that the sub-audio corresponding to the sub-text belongs to various violation types.

[0011] Based on the target violation label vector of each sub-text, a target violation type label is determined for each sub-audio segment, wherein the target violation type label indicates whether the sub-audio is a violation audio or a non-violation audio.

[0012] In an optional implementation, the violation type includes no violation and various other violations, and the target violation label vector includes multiple probability values, each probability value corresponding to a violation type; the step of determining the violation type label for each sub-audio segment based on the target violation label vector of each sub-text includes:

[0013] For each sub-audio segment, determine the maximum probability value in the target violation tag vector of the sub-text corresponding to the sub-audio segment;

[0014] The violation type corresponding to the maximum probability value is used as the violation type label for the sub-audio.

[0015] In an optional implementation, the violation review model is trained in the following manner:

[0016] Construct N training subsets, each of which includes at least one training audio corresponding to multiple training sub-texts. Each training sub-text has a violation type label, and the violation type label represents the actual violation type of the training audio to which the training sub-text belongs.

[0017] The pre-built violation review model is trained using N training subsets to obtain the trained violation review model.

[0018] In an optional implementation, the step of constructing N training subsets includes:

[0019] Obtain several training audio files and the violation type label for each training audio file;

[0020] Speech recognition is performed on each of the training audio samples to obtain several ASR texts;

[0021] Based on the preset splitting rules, each ASR text is split into multiple training sub-texts, and the violation type label of each training audio is used as the violation type label of each training sub-text corresponding to the training audio.

[0022] All the training sub-texts are grouped to obtain the N training subsets; wherein all training sub-texts belonging to the same training audio are located in one training subset.

[0023] In an optional implementation, the step of training a pre-built violation review model using N training subsets to obtain the trained violation review model includes:

[0024] The N training subsets are sorted out of order to obtain the N training subsets after scrambling.

[0025] Obtain the target probability vector for each training text in the current training subset; the current training subset is the first of the N training subsets after scrambling; the target probability vector represents the violation type label of the training text.

[0026] Using the current training subset and the target probability vector of each training text in the current training subset, the violation review model is trained to obtain the adjusted violation review model;

[0027] Determine whether the adjusted violation review model meets the training stop condition;

[0028] If the conditions are not met, the next training subset is input into the adjusted violation review model to obtain the pseudo-label vector of each training text in the next training subset; the violation types include no violation and various other violations, and the pseudo-label vector includes multiple predicted probability values, each predicted probability value corresponding to a violation type;

[0029] For each of the pseudo-label vectors, the pseudo-label vectors are processed to obtain the target probability vector of the corresponding training sub-text;

[0030] The next training subset is used as the current training subset, and the step of training the violation review model using the current training subset and the target probability vector of each training subtext in the current training subset to obtain the adjusted violation review model is returned until the adjusted violation review model meets the training stopping condition, thus obtaining the trained violation review model.

[0031] In an optional implementation, the training stopping condition is that the adjusted violation review model has been trained for a set number of rounds using the N training subsets, or that the cost function curve containing the loss function value calculated after the validation set is input into the adjusted violation review model reaches an inflection point.

[0032] Secondly, the present invention provides an audio segmentation violation review device, comprising:

[0033] The data acquisition module is used for:

[0034] Acquire the audio to be processed, and perform speech recognition on the audio to be processed to obtain the ASR text to be processed;

[0035] Based on preset splitting rules, the ASR text to be processed is split into multiple sub-texts; each sub-text corresponds to a segment of the audio to be processed.

[0036] The data processing module is used for:

[0037] Each of the sub-texts is input into the trained violation review model to obtain a target violation label vector for each sub-text. The target violation label vector uses different values ​​to represent the probability that the sub-audio corresponding to the sub-text belongs to various violation types.

[0038] Based on the target violation label vector of each sub-text, a target violation type label is determined for each sub-audio segment, wherein the target violation type label indicates whether the sub-audio is a violation audio or a non-violation audio.

[0039] In an optional implementation, the violation types include no violation and various other violations. The target violation label vector includes multiple probability values, each probability value corresponding to a violation type. The data processing module, when determining the violation type label for each segment of sub-audio based on the target violation label vector of each sub-text, specifically performs the following:

[0040] For each sub-audio segment, determine the maximum probability value in the target violation tag vector of the sub-text corresponding to the sub-audio segment;

[0041] The violation type corresponding to the maximum probability value is used as the violation type label for the sub-audio.

[0042] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to implement the audio segmentation violation review method as described in any of the foregoing embodiments.

[0043] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio segmentation violation review method described in any of the foregoing embodiments.

[0044] Compared with existing technologies, this invention provides a method, apparatus, electronic device, and storage medium for segmented audio violation review. It obtains the ASR text to be processed from the audio to be processed, and then, based on preset segmentation rules, segments the ASR text to be processed into multiple sub-texts corresponding to the sub-audio segments of the audio to be processed. Next, each sub-text is input into a trained violation review model to obtain a target violation label vector for each sub-text. Different values ​​in the target violation label vector represent the probability that the corresponding sub-audio segment belongs to various violation types. Finally, based on the target violation label vector of each sub-text, the target violation type label for each sub-audio segment is determined. The target violation type label indicates whether the sub-audio segment is a violation or not. Thus, through automated review, violation audio can be reviewed promptly and effectively, and the segmented review method of this solution can accurately locate small segments of violation sub-audio segments within a longer audio to be processed. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0047] Figure 2 This is one of the flowcharts illustrating an audio segmentation violation review method provided in an embodiment of the present invention.

[0048] Figure 3 This is the second flowchart illustrating a method for segmenting and reviewing audio violations, as provided in an embodiment of the present invention.

[0049] Figure 4 This is the third flowchart illustrating an audio segmentation violation review method provided in an embodiment of the present invention.

[0050] Figure 5 This is a schematic diagram of the structure of an audio segmentation violation review device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0053] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0054] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0055] The following describes in detail the audio segmentation violation review method provided by the present invention through embodiments and in conjunction with the accompanying drawings.

[0056] Please see Figure 1 , Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes a processor 110, a memory 120, and a bus 130, with the processor 110 connected to the memory 120 via the bus 130.

[0057] The memory 120 can be used to store software programs and modules, such as the program instructions / modules corresponding to the audio segmentation violation review device 200 provided in the embodiments of the present invention. The processor 110 executes various functional applications and data processing by running the software programs and modules stored in the memory 120, such as the audio segmentation violation review method provided in the embodiments of the present invention.

[0058] The memory 120 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0059] The processor 110 can be an integrated circuit chip with signal processing capabilities. The processor 110 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0060] Optionally, the electronic device 100 may be, but is not limited to, a smartphone, a smart tablet, a personal computer, a server, etc.

[0061] Understandable. Figure 1 The structure shown is for illustrative purposes only; the electronic device 100 may also include components that are more advanced than those shown. Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0062] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating a method for segmented violation review of audio provided in an embodiment of the present invention. The executing entity of this method can be the aforementioned electronic device, and the method includes the following steps:

[0063] S300: Acquire the audio to be processed and perform speech recognition on the audio to be processed to obtain the ASR text to be processed.

[0064] S400: Based on preset splitting rules, split the ASR text to be processed to obtain multiple sub-texts.

[0065] In this embodiment, the preset splitting rule can be to split the ASR text to be processed every set number of characters to obtain a sub-text; or it can be to split the audio to be processed based on the position of the splitting node corresponding to the set duration in the ASR text to obtain a sub-text. Therefore, the audio to be processed is also regarded as multiple sub-audio segments, and each sub-audio segment has a corresponding sub-text.

[0066] In the optional examples, the word count or duration settings are based on actual situations. For instance, for audio files of 30 minutes or more, to ensure the accuracy of the review process, the word count can be set to 200 or 300, or the duration to one or two minutes. Please understand that these examples are for illustrative purposes only and are not intended to be limiting.

[0067] S500. Input each subtext into the trained violation review model to obtain the target violation label vector for each subtext.

[0068] In this embodiment, the violation types include no violation and various other violations. The target violation label vector can include multiple vector values ​​that sum to 1. Each vector value corresponds to a violation type. The target violation label vector uses different values ​​to represent the probability that the sub-audio corresponding to the sub-text belongs to various violation types.

[0069] S600. Based on the target violation label vector of each sub-text, determine the target violation type label for each sub-audio segment.

[0070] In this embodiment, the target violation type label indicates whether the sub-audio is a violation audio or a non-violation audio.

[0071] The audio segmentation violation review method provided in this invention obtains the ASR text to be processed from the audio to be processed, and then, based on preset segmentation rules, segments the ASR text to be processed into multiple sub-texts corresponding to the sub-audio segments of the audio to be processed. Next, each sub-text is input into a trained violation review model to obtain a target violation label vector for each sub-text. Different values ​​in the target violation label vector represent the probability that the corresponding sub-audio segment belongs to various violation types. Finally, based on the target violation label vector of each sub-text, the target violation type label for each sub-audio segment is determined. The target violation type label indicates whether the sub-audio segment is a violation-related audio segment or not. Thus, the automated review method can promptly and effectively identify violation-related audio, and the segmentation review method of this solution can accurately locate small segments of violation-related sub-audio segments within a longer audio to be processed.

[0072] In an optional implementation, the violation type label of the sub-audio can be determined to be the maximum value, and the sub-steps of step S600 above may include:

[0073] S610. For each sub-audio segment, determine the maximum probability value in the target violation tag vector of the sub-text corresponding to the sub-audio segment.

[0074] S620. Use the violation type corresponding to the highest probability value as the violation type label for the sub-audio.

[0075] In the optional example, it is assumed that the violation type includes no violation and other violations (V1, V2, V3), and the audio to be processed S includes sub-audio S1 to S10.

[0076] If the target violation label vector of sub-audio S1 is [0.1, 0.7, 0.1, 0.1], then the maximum probability value is 0.7, and the violation type corresponding to 0.7 is V1. Therefore, the target violation type label of sub-audio S1 is V1, which belongs to the category of violation audio.

[0077] If the target violation label vector of sub-audio S2 is [0.8, 0.05, 0.1, 0.05], then the violation type corresponding to the maximum probability value of 0.8 is non-violation. Therefore, the target violation type label of sub-audio S2 is non-violation, and it belongs to non-violation audio.

[0078] It should be noted that the above examples are merely illustrative and are not intended to be limiting.

[0079] The training process of the aforementioned violation review model will be described next.

[0080] In an optional implementation, the violation review model can employ a multi-classification model, such as the BERT model. Please refer to... Figure 3 The model training process may include the following steps S100 to S200:

[0081] S100, Construct N training subsets.

[0082] In this embodiment, each training subset may include at least one training audio corresponding to multiple training sub-texts. Each training sub-text may have a violation type label, which can characterize the actual violation type of the training audio to which the training sub-text belongs.

[0083] In an optional example, the sub-steps of S100 may include:

[0084] S110, Obtain several training audios and the violation type label for each training audio.

[0085] In this embodiment, the actual violation category of the training audio can be the review result obtained from manual review.

[0086] S120. Perform speech recognition on each training audio to obtain several ASR texts.

[0087] S130. Based on the preset splitting rules, each ASR text is split into multiple training sub-texts, and the violation type label of each training audio is used as the violation type label of each training sub-text corresponding to the training audio.

[0088] S140. Divide all training subtexts into groups to obtain N training subsets.

[0089] In this embodiment, all training subtexts belonging to the same training audio can be located in a training subset.

[0090] S200. Train the pre-built violation review model using N training subsets to obtain the trained violation review model.

[0091] In the optional examples, please combine Figure 4 The sub-steps of S200 may include:

[0092] S210. Randomize the N training subsets to obtain the N training subsets after randomization.

[0093] S220. Obtain the target probability vector for each training text in the current training subset.

[0094] In this embodiment, the current training subset is the first of the N training subsets after scrambling. The target probability vector can include the vector value corresponding to each violation type. Among the various violation types, the vector value corresponding to the actual violation type of the training subtext is 1, while the vector values ​​corresponding to the other violation types are all 0. Therefore, the target probability vector can represent the violation type label of the training subtext.

[0095] For example, assuming that the violation types include no violation and other violations (V1, V2, V3), and the violation type label of a certain training subtext is V3, then the target probability vector of the training subtext is [0, 0, 0, 1].

[0096] It should be noted that only in the first training subset during the first round of training is the target probability vector for each training text obtained based on the violation type label of the training text. In subsequent training processes, the target probability vector for each training text is obtained by processing the pseudo-label vector output by the model.

[0097] S230. Using the current training subset and the target probability vector of each training text in the current training subset, train the violation review model to obtain the adjusted violation review model.

[0098] In this embodiment, all training texts of the current training subset are first input into the violation review model to obtain the predicted violation label vector for each training text. Then, for each training text in the current training subset: the predicted violation label vector and the target probability vector of that training text are substituted into a preset loss function to calculate an undetermined loss vector. The sum of all values ​​in this undetermined loss vector is the loss value corresponding to that training text. Thus, the loss value for each training text in the current training subset is obtained. Then, the loss values ​​for each training text are summed to obtain the total loss value for the current training subset. The total loss value is used for backpropagation to update the model parameters of the violation review model, resulting in the adjusted violation review model.

[0099] In this context, each vector value of the undetermined loss vector represents the loss value corresponding to a violation category. In an optional example, the preset loss function can be multi-class cross-entropy.

[0100] S240. Determine whether the adjusted violation review model meets the training stop condition.

[0101] In this embodiment, if the adjusted violation review model does not meet the training stop condition, the following step S250 is executed; if the adjusted violation review model meets the training stop condition, it means that the training is complete and the trained violation review model has been obtained.

[0102] S250. Input the next training subset into the adjusted violation review model to obtain the pseudo-label vector of each training text in the next training subset.

[0103] In this embodiment, the violation types include no violation and various other violations. The pseudo-label vector includes multiple predicted probability values ​​that sum to 1, and each predicted probability value corresponds to a violation type. In the N training subsets after scrambling, the current training subset is the nth subset, and the next training subset is the (n+1)%Nth subset.

[0104] For example, assuming N is 10, that is, a total of 10 training subsets [T0, T1, T2, ..., T9], then if the current training subset is T0, the next training subset will be T1; if the current training subset is T9, the next training subset will be T0.

[0105] Specifically, if the current data subset is the last of the N training subsets, it means that the violation review model has completed one round of training on all training data. In this case, to improve the randomness of the data, the N training subsets can be shuffled again, and then the next training subset can be input into the adjusted violation review model.

[0106] S260. For each pseudo-label vector, process the pseudo-label vector to obtain the target probability vector of the corresponding training sub-text, and take the next training subset as the current training subset.

[0107] In this embodiment, after executing step S260, the process can return to executing step S230 until the adjusted violation review model meets the training stop condition, which signifies that the training is complete and the trained violation review model is obtained.

[0108] In the optional examples, the training stopping condition can be one of the following two:

[0109] First, the adjusted violation review model is trained for a set number of rounds using N training subsets. This set number of rounds, m, can be configured according to actual needs.

[0110] Second, the cost function curve containing the loss function value calculated after inputting the adjusted violation review model into the validation set shows an inflection point.

[0111] It's understandable that after training a subset of data to obtain an adjusted violation review model, a validation set can be input into this adjusted model to obtain a cost function value. These multiple cost function values ​​can then be plotted to create a cost function curve. As the number of training iterations increases, the cost function curve will eventually reach an inflection point, gradually flattening out from a gradual increase / decrease.

[0112] In an optional example, the target probability vector can be obtained by processing the pseudo-label vector of a training subtext by generating random numbers. This process may include the following steps:

[0113] S261. Generate a random number between 0 and 1;

[0114] S262. Compare the random number with the target vector value corresponding to the actual violation type of the training subtext in the pseudo-label vector;

[0115] S263. If the random number is less than the target vector value, then set the target vector value in the pseudo-label vector to 1 and set all other vector values ​​to 0 to obtain the target probability vector of the training subtext.

[0116] S264. If the random number is greater than or equal to the target vector value, then set the vector value corresponding to the non-violation in the pseudo-label vector to 1, and set all other vector values ​​to 0, to obtain the target probability vector of the training subtext.

[0117] The following examples illustrate this.

[0118] Assuming the violation types include no violation and other violations (V1, V2, V3), and the actual violation type of a training sub-text K is V1, the pseudo-label vector of this training sub-text K is [0.05, 0.8, 0.05, 0.1]. Then, the target vector value in the pseudo-label vector is 0.8. Generating a random number between 0 and 1, the target probability vector obtained based on this random number can have the following two possibilities:

[0119] First type: The random number is less than the target vector value.

[0120] For example, when the random number is 0.2, since 0.2 < 0.8, the value of 0.8 in the pseudo-label vector is set to 1, and the other three vector values ​​are set to 0. That is, the pseudo-label vector [0.05, 0.1, 0.05, 0.8] of the training subtext K is converted into the target probability vector [0, 1, 0, 0].

[0121] The second method is to have a random number greater than or equal to the target vector value:

[0122] For example, when the random number is 0.9, since 0.9 > 0.8, the 0.05 value corresponding to the non-violation in the pseudo-label vector is set to 1, and the other three vector values ​​are directly set to 0. That is, the pseudo-label vector [0.05, 0.1, 0.05, 0.8] of the training subtext K is converted into the target probability vector [1, 0, 0, 0].

[0123] It should be noted that the above examples are merely illustrative and are not intended to be limiting.

[0124] It is understandable that the actual violation category of the entire training audio is certain; however, the actual violation category of each training sub-text corresponding to the training audio is uncertain, because it is not certain which training sub-text contains the actual violation category.

[0125] Therefore, for the entire training audio, the actual violation category is definite, while other violation types are indefinite. However, for each training sub-text, each violation type is indefinite, and the violation type label of the training sub-text may be an incorrect label. Thus, at the beginning of training, the target probability vector for each training sub-text in the first training subset is inaccurate. However, in subsequent training, the target probability vector for each training sub-text is obtained by processing the pseudo-label vector output by the model. Using the pseudo-label vector can correct the influence of incorrect labels, thereby allowing the model to become more accurate as training progresses.

[0126] It should be noted that the execution order of each step in the above method embodiments is not limited to that shown in the attached figures, and the execution order of each step shall be subject to the actual application situation.

[0127] In order to perform the corresponding steps in the above method embodiments and various possible implementations, the following provides an implementation method of an audio segmentation violation review device.

[0128] Please see Figure 5 , Figure 5 A schematic diagram of the structure of an audio segmentation violation review device provided in an embodiment of the present invention is shown. The audio segmentation violation review device 200 includes: a data acquisition module 230 and a data processing module 240.

[0129] Data acquisition module 230 is used for:

[0130] The audio to be processed is acquired, and speech recognition is performed on the audio to be processed to obtain the ASR text to be processed; based on the preset splitting rules, the ASR text to be processed is split into multiple sub-texts; each sub-text corresponds to a segment of the audio to be processed.

[0131] Data processing module 240 is used for:

[0132] Each subtext is input into the trained violation review model to obtain the target violation label vector for each subtext. The target violation label vector uses different values ​​to represent the probability that the corresponding audio segment belongs to various violation types. Based on the target violation label vector of each subtext, the target violation type label for each audio segment is determined. The target violation type label represents whether the audio segment is a violation audio or a non-violation audio.

[0133] In an optional implementation, the violation type includes no violation and various other violations, and the target violation label vector includes multiple probability values, each probability value corresponding to a violation type. The data processing module 240, when determining the violation type label for each sub-audio segment based on the target violation label vector of each sub-text, specifically performs the following: for each sub-audio segment, determine the maximum probability value in the target violation label vector of the sub-text corresponding to the sub-audio segment; and use the violation type corresponding to the maximum probability value as the violation type label of the sub-audio segment.

[0134] In an optional implementation, the audio segmentation violation review device 200 further includes:

[0135] Data collection module 210 is used to construct N training subsets. Each training subset includes at least one training audio corresponding to multiple training subtexts. Each training subtext has a violation type label. The violation category label represents the actual violation type of the training audio to which the training subtext belongs.

[0136] The model training module 220 is used to train a pre-built violation review model using N training subsets to obtain the trained violation review model.

[0137] In an optional implementation, when the data collection module 210 is used to construct N training subsets, it can specifically be used to: acquire several training audios and the violation type label of each training audio; perform speech recognition on each training audio to obtain several ASR texts; based on a preset splitting rule, split each ASR text into multiple training sub-texts, and use the violation type label of each training audio as the violation type label of each training sub-text corresponding to the training audio; group all training sub-texts to obtain N training subsets; wherein all training sub-texts belonging to the same training audio are located in one training subset.

[0138] In an optional implementation, the model training module 220 is used to train a pre-built violation review model using N training subsets. When the trained violation review model is obtained, it can be specifically used to: sort the N training subsets in random order to obtain N training subsets in random order.

[0139] Obtain the target probability vector for each training text in the current training subset; the current training subset is the first of the N training subsets after scrambling; the target probability vector represents the violation type label of the training text;

[0140] Using the current training subset and the target probability vector of each training text in the current training subset, train the violation review model to obtain the adjusted violation review model;

[0141] Determine whether the adjusted violation review model meets the training termination condition;

[0142] If the conditions are not met, the next training subset is input into the adjusted violation review model to obtain the pseudo-label vector for each training text in the next training subset; the violation types include no violation and various other violations, and the pseudo-label vector includes multiple predicted probability values, each of which corresponds to a violation type;

[0143] For each pseudo-label vector, the pseudo-label vector is processed to obtain the target probability vector of the corresponding training sub-text;

[0144] The next training subset is used as the current training subset, and the process of training the violation review model using the current training subset and the target probability vector of each training text in the current training subset is returned to obtain the adjusted violation review model. This process continues until the adjusted violation review model meets the training stopping condition, thus obtaining the trained violation review model.

[0145] In an optional implementation, the training stopping condition is that the adjusted violation review model has been trained for a set number of rounds using N training subsets, or that the cost function curve containing the loss function value calculated after the validation set is input into the adjusted violation review model reaches an inflection point.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the audio segmentation violation review device 200 described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0147] This invention also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the audio segmentation violation review method disclosed in the above embodiments. The computer-readable storage medium can be, but is not limited to, various media capable of storing program code, such as a USB flash drive, external hard drive, ROM, RAM, PROM, EPROM, EEPROM, FLASH disk, or optical disk.

[0148] In summary, this invention provides a method, apparatus, electronic device, and storage medium for segmented audio violation review. It obtains the ASR text of the audio to be processed, and then, based on preset segmentation rules, segments the ASR text to obtain multiple sub-texts corresponding to the sub-audio segments of the audio to be processed. Next, each sub-text is input into a trained violation review model to obtain a target violation label vector for each sub-text. Different values ​​in the target violation label vector represent the probability that the corresponding sub-audio segment belongs to various violation types. Finally, based on the target violation label vector of each sub-text, the target violation type label for each sub-audio segment is determined. The target violation type label indicates whether the sub-audio segment is a violation or not. Thus, through automated review, violation audio can be reviewed promptly and effectively, and the segmented review method of this solution can accurately locate small segments of violation sub-audio segments within a longer audio file.

[0149] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for segmented violation review of audio, characterized in that, include: Acquire the audio to be processed, and perform speech recognition on the audio to be processed to obtain the ASR text to be processed; Based on preset splitting rules, the ASR text to be processed is split into multiple sub-texts; each sub-text corresponds to a segment of the audio to be processed. Each of the sub-texts is input into the trained violation review model to obtain a target violation label vector for each sub-text. The target violation label vector uses different values ​​to represent the probability that the sub-audio corresponding to the sub-text belongs to various violation types. Based on the target violation label vector of each sub-text, a target violation type label is determined for each sub-audio segment, wherein the target violation type label indicates whether the sub-audio is a violation audio or a non-violation audio; The trained violation review model was obtained through the following method: Obtain several training audio files and a violation type label for each training audio file; the violation type label represents the actual violation type identified in the training audio file. Speech recognition is performed on each of the training audio samples to obtain several ASR texts; Based on the preset splitting rules, each ASR text is split into multiple training sub-texts, and the violation type label of each training audio is used as the violation type label of each training sub-text corresponding to the training audio. All the training sub-texts are grouped to obtain N training subsets; where all training sub-texts belonging to the same training audio are located in one training subset. The N training subsets are sorted out of order to obtain the N training subsets after scrambling. The first of the N disordered training subsets is taken as the current training subset, and the target probability vector of each training text in the current training subset is obtained; the target probability vector represents the violation type label of the training text. Using the current training subset and the target probability vector of each training text in the current training subset, the violation review model is trained to obtain the adjusted violation review model; Determine whether the adjusted violation review model meets the training stop condition; If the conditions are not met, the next training subset is input into the adjusted violation review model to obtain the pseudo-label vector of each training text in the next training subset; the violation types include no violation and various other violations, and the pseudo-label vector includes multiple predicted probability values, each predicted probability value corresponding to a violation type; For each of the pseudo-label vectors, the pseudo-label vectors are processed to obtain the target probability vector of the corresponding training sub-text; The next training subset is used as the current training subset, and the step of training the violation review model using the current training subset and the target probability vector of each training subtext in the current training subset to obtain the adjusted violation review model is returned until the adjusted violation review model meets the training stopping condition, and the trained violation review model is obtained. The step of processing the pseudo-label vector to obtain the target probability vector of the corresponding training sub-text includes: Generate a random number between 0 and 1, and compare the random number with the target vector value corresponding to the actual violation type of the training subtext in the pseudo-label vector; If the random number is less than the target vector value, then the target vector value in the pseudo-label vector is set to 1, and all other vector values ​​are set to 0, to obtain the target probability vector of the training subtext. If the random number is greater than or equal to the target vector value, then the vector value corresponding to the non-violation in the pseudo-label vector is set to 1, and all other vector values ​​are set to 0, to obtain the target probability vector of the training subtext.

2. The method of claim 1, wherein, The violation types include no violation and various other violations. The target violation label vector includes multiple probability values, each probability value corresponding to a violation type. The step of determining the violation type label for each sub-audio segment based on the target violation label vector of each sub-text includes: For each sub-audio segment, determine the maximum probability value in the target violation tag vector of the sub-text corresponding to the sub-audio segment; The violation type corresponding to the maximum probability value is used as the violation type label for the sub-audio.

3. The method of claim 1, wherein, The training stopping condition is that the adjusted violation review model has been trained for a set number of rounds using the N training subsets, or that the cost function curve containing the loss function value calculated after the validation set is input into the adjusted violation review model reaches an inflection point.

4. An audio segmentation violation review device, characterized in that, include: The data acquisition module is used for: Acquire the audio to be processed, and perform speech recognition on the audio to be processed to obtain the ASR text to be processed; Based on preset splitting rules, the ASR text to be processed is split into multiple sub-texts; each sub-text corresponds to a segment of the audio to be processed. The data processing module is used for: Each of the sub-texts is input into the trained violation review model to obtain a target violation label vector for each sub-text. The target violation label vector uses different values ​​to represent the probability that the sub-audio corresponding to the sub-text belongs to various violation types. Based on the target violation label vector of each sub-text, a target violation type label is determined for each sub-audio segment, wherein the target violation type label indicates whether the sub-audio is a violation audio or a non-violation audio; The device further includes a data collection module and a model training module: The data collection module is used for: Obtain several training audio files and a violation type label for each training audio file; the violation type label represents the actual violation type identified in the training audio file. Speech recognition is performed on each of the training audio samples to obtain several ASR texts; Based on the preset splitting rules, each ASR text is split into multiple training sub-texts, and the violation type label of each training audio is used as the violation type label of each training sub-text corresponding to the training audio. All the training sub-texts are grouped to obtain N training subsets; where all training sub-texts belonging to the same training audio are located in one training subset. The model training module is used for: The N training subsets are sorted out of order to obtain the N training subsets after scrambling. The first of the N disordered training subsets is taken as the current training subset, and the target probability vector of each training text in the current training subset is obtained; the target probability vector represents the violation type label of the training text. Using the current training subset and the target probability vector of each training text in the current training subset, the violation review model is trained to obtain the adjusted violation review model; Determine whether the adjusted violation review model meets the training stop condition; If the conditions are not met, the next training subset is input into the adjusted violation review model to obtain the pseudo-label vector of each training text in the next training subset; the violation types include no violation and various other violations, and the pseudo-label vector includes multiple predicted probability values, each predicted probability value corresponding to a violation type; For each of the pseudo-label vectors, the pseudo-label vectors are processed to obtain the target probability vector of the corresponding training sub-text; The next training subset is used as the current training subset, and the step of training the violation review model using the current training subset and the target probability vector of each training subtext in the current training subset to obtain the adjusted violation review model is returned until the adjusted violation review model meets the training stopping condition, and the trained violation review model is obtained. Specifically, when the model training module processes the pseudo-label vector to obtain the target probability vector of the corresponding training sub-text, it is used for: Generate a random number between 0 and 1, and compare the random number with the target vector value corresponding to the actual violation type of the training subtext in the pseudo-label vector; If the random number is less than the target vector value, then the target vector value in the pseudo-label vector is set to 1, and all other vector values ​​are set to 0, to obtain the target probability vector of the training subtext. If the random number is greater than or equal to the target vector value, then the vector value corresponding to the non-violation in the pseudo-label vector is set to 1, and all other vector values ​​are set to 0, to obtain the target probability vector of the training subtext.

5. The apparatus of claim 4, wherein, Violation types include no violation and various other violations. The target violation label vector includes multiple probability values, each probability value corresponding to a violation type. The data processing module, when determining the violation type label for each segment of audio based on the target violation label vector of each sub-text, specifically performs the following: For each sub-audio segment, determine the maximum probability value in the target violation tag vector of the sub-text corresponding to the sub-audio segment; The violation type corresponding to the maximum probability value is used as the violation type label for the sub-audio.

6. An electronic device, comprising: include: The electronic device includes a memory and a processor, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is running, execute the machine-readable instructions to implement the audio segmentation violation review method as described in any one of claims 1-3.

7. A computer readable storage medium characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the segmented violation review method for audio as described in any one of claims 1-3.

Images