Audio violation review method, device, electronic device and readable storage medium

Abstract

The present invention provides a method, device, electronic device and readable storage medium for audio violation review, which relates to the field of artificial intelligence technology. When training a multi-label classification model, a training sample set and an auxiliary data set are used. The training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio; the violation mask vector uses different numerical values ​​to represent the actual violation category and other pending violation categories of the training audio; the target probability vector includes the probability that the training audio belongs to the actual violation category. In this way, the target loss value obtained by using the violation mask vector during training only retains the influence of the actual violation category, making the multi-label classification model more accurate, so that all target violation categories to which the audio to be reviewed belongs can be determined in the model application stage, ensuring the comprehensiveness of violation category identification.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and readable storage medium for audio violation review. Background Art

[0002] With the development of the internet, online social platforms are also booming. People can share various things on online platforms, such as text sharing on social media platforms and audio sharing on audio sharing platforms. However, not all shared content can be published on the platform. Shared content must first be reviewed for violations and confirmed to be free of any violations before it can be published.

[0003] Traditional manual review relies on backend reviewers to manually conduct judgment and review. This method is time-consuming, inefficient, and requires significant human, material, and financial resources. For example, for user-shared audio, manual review involves playing the audio and reviewers listening to determine if there is any illegal content. If any illegal content is detected, the audio is labeled as illegal and rejected for publication.

[0004] However, in reality, the reviewer may not have listened to the entire audio, and there are many types of violations. The part that the reviewer did not listen to may not contain any violations or may contain other types of violations. The violation review results given in this way are too one-sided. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device, electronic device and readable storage medium for audio violation review to improve the problems existing in the prior art.

[0006] The embodiments of the present invention can be implemented as follows:

[0007] In a first aspect, the present invention provides a method for reviewing audio violations, comprising:

[0008] Acquire the audio to be reviewed, and perform speech recognition on the audio to be reviewed to obtain the ASR text to be reviewed;

[0009] Inputting the ASR text to be reviewed into a trained multi-label classification model to obtain a violation label prediction vector, wherein the violation label prediction vector includes a probability value of the audio to be reviewed belonging to each violation category;

[0010] Determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector;

[0011] The multi-label classification model is trained using a training sample set and an auxiliary data set, wherein the training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio;

[0012] The violation mask vector uses different values ​​to represent the actual violation category determined for the training audio and other pending violation categories; the target probability vector includes the probability that the training audio belongs to the actual violation category.

[0013] In an optional embodiment, the multi-label classification model is trained in the following manner:

[0014] Constructing the training sample set and the auxiliary data set;

[0015] The pre-built multi-label classification model is trained using the training sample set and the auxiliary data set to obtain the trained multi-label classification model.

[0016] In an optional embodiment, the step of constructing the training sample set and the auxiliary data set includes:

[0017] Acquire a plurality of training audios; the training audios are provided with violation category labels, wherein the violation category labels represent actual violation categories determined for the training audios;

[0018] Performing speech recognition on each of the training audios to obtain a plurality of ASR texts to constitute the training sample set;

[0019] For each of the training audios, the violation mask vector and target probability vector of the training audio are determined based on the violation category label of the training audio and multiple preset violation categories, and the violation mask vector and target probability vector of each of the training audios are obtained to constitute the auxiliary data set; wherein, the target probability vector represents that the probability that the training audio belongs to the actual violation category is 1, and the probabilities of belonging to other pending violation categories are all set parameters.

[0020] In an optional embodiment, the multiple violation categories include audio non-violation and multiple other violation categories; the violation mask vector includes a vector value corresponding to each violation category;

[0021] The step of determining the violation mask vector of the training audio based on the violation category label of the training audio and multiple preset violation categories includes:

[0022] When the actual violation category is no audio violation, the vector value corresponding to the actual violation category and the vector values ​​corresponding to multiple other violation categories in the violation Mask vector are all set to 1;

[0023] When the actual violation category is one of the multiple other violation categories, the vector values ​​corresponding to the actual violation category and the audio non-violation in the violation mask vector are all set to 1, and the vector values ​​corresponding to the pending violation categories other than the actual violation category in the multiple other violation categories are all set to 0.

[0024] In an optional embodiment, the step of training a pre-built multi-label classification model using the training sample set and the auxiliary data set includes:

[0025] Inputting the training sample set into the multi-label classification model to obtain a prediction probability vector for each ASR text, wherein the prediction probability vector includes a probability value corresponding to each violation category;

[0026] For each ASR text, calculate the individual loss value of the ASR text using the predicted probability vector, target probability vector and violation Mask vector of the ASR text;

[0027] Get the target loss value based on the individual loss value of each ASR text;

[0028] Backpropagation is performed using the target loss value to update model parameters of the multi-label classification model to obtain the trained multi-label classification model.

[0029] In an optional embodiment, the step of calculating the individual loss value of the ASR text using the prediction probability vector, the target probability vector, and the violation Mask vector of the ASR text includes:

[0030] Calculating a pending loss vector based on the target probability vector, the predicted probability vector, and a preset loss function;

[0031] Multiplying the pending loss vector by the violation Mask vector to obtain a target loss vector;

[0032] Each loss value of the target loss vector is added together to obtain the individual loss value.

[0033] In a second aspect, the present invention provides an audio violation review device, comprising:

[0034] An acquisition module is used to acquire the audio to be reviewed and perform speech recognition on the audio to be reviewed to obtain the ASR text to be reviewed;

[0035] A prediction module is configured to input the ASR text to be reviewed into a trained multi-label classification model to obtain a violation label prediction vector, wherein the violation label prediction vector includes a probability value of the audio to be reviewed belonging to each violation category;

[0036] Determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector;

[0037] The multi-label classification model is trained using a training sample set and an auxiliary data set. The training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio. The violation mask vector uses different numerical values ​​to represent the actual violation category and other pending violation categories of the training audio. The target probability vector includes the probability that the training audio belongs to the actual violation category.

[0038] In an optional embodiment, the device further comprises:

[0039] The training module is used to: construct the training sample set and the auxiliary data set; and train a pre-constructed multi-label classification model using the training sample set and the auxiliary data set to obtain the trained multi-label classification model.

[0040] In a third aspect, the present invention provides an electronic device comprising: a memory and a processor, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to implement the audio violation review method as described in any one of the aforementioned embodiments.

[0041] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the audio violation review method as described in any one of the aforementioned implementation methods.

[0042] Compared with the prior art, the embodiments of the present invention provide a method, device, electronic device and readable storage medium for reviewing violations of audio. When training a multi-label classification model, a training sample set and an auxiliary data set are used. The training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio. The violation mask vector uses different numerical values ​​to represent the actual violation category and other pending violation categories of the training audio. The target probability vector includes the probability that the training audio belongs to the actual violation category. In this way, the target loss value obtained using the violation mask vector during training only retains the influence of the actual violation category, making the multi-label classification model more accurate, so that all target violation categories to which the audio to be reviewed belongs can be determined during the model application stage, ensuring the comprehensiveness of violation category identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0045] Figure 2 One of the flow charts of an audio violation review method provided in an embodiment of the present invention.

[0046] Figure 3 The present invention provides a second flowchart of a method for reviewing audio violations in accordance with an embodiment of the present invention.

[0047] Figure 4 A schematic diagram of the structure of an audio violation review device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not require further definition or explanation in subsequent drawings.

[0051] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0052] The following describes in detail the audio violation review provided by the embodiment of the present invention through examples and in conjunction with the attached drawings.

[0053] See Figure 1 , Figure 1This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 100 includes a processor 110 , a memory 120 , and a bus 130 , wherein the processor 110 is connected to the memory 120 via the bus 130 .

[0054] The memory 120 can be used to store software programs and modules, such as program instructions / modules corresponding to the audio violation review device 200 provided in an embodiment of the present invention. The processor 110 executes the software programs and modules stored in the memory 120 to perform various functional applications and data processing, such as the audio violation review method provided in an embodiment of the present invention.

[0055] Among them, the memory 120 can be, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory (Flash), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0056] The processor 110 may be an integrated circuit chip with signal processing capabilities. The processor 110 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0057] Optionally, the electronic device 100 may be, but is not limited to, a personal computer, a smart phone, a server, etc.

[0058] I understand. Figure 1 The structure shown is for illustration only. The electronic device 100 may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0059] Please refer to Figure 2 , Figure 2 This is a flow chart of a method for reviewing audio violations provided by an embodiment of the present invention. The method may be performed by the aforementioned electronic device. The method includes the following steps:

[0060] S210: Acquire the audio to be reviewed, perform speech recognition on the audio to be reviewed, and obtain the ASR text to be reviewed.

[0061] S220 , input the ASR text to be reviewed into the trained multi-label classification model to obtain a violation label prediction vector.

[0062] In this embodiment, the violation label prediction vector includes the probability that the audio to be reviewed belongs to each violation category. The multi-label classification model can be trained using a training sample set and an auxiliary dataset. The training sample set includes the ASR text of several training audios, and the auxiliary dataset includes the target probability vector and violation mask vector for each training audio.

[0063] The violation mask vector is a binary vector that uses different values ​​to represent the actual violation category determined in the training audio and other pending violation categories. The target probability vector contains the probability that the training audio belongs to the actual violation category.

[0064] Optionally, multiple violation categories may be preset, one of which indicates that the audio does not violate a violation. The violation label prediction vector may include a vector value corresponding to each violation category, where the vector value represents the probability that the audio to be reviewed belongs to the corresponding violation category.

[0065] S230: Determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector.

[0066] In this embodiment, all target violation categories to which the audio to be reviewed belongs can be determined based on a pre-set comparison threshold. Specifically, each vector value in the violation label prediction vector is compared with the comparison threshold. If there is a vector value greater than the comparison threshold, then it can be determined that the violation category corresponding to the vector value is a target violation category existing in the audio to be reviewed.

[0067] In a possible example, assume that multiple violation categories include A1, A2, A3, A4, and audio without violation. Corresponding to these five violation categories, the violation label prediction vector of the audio to be reviewed is [0.4, 0.8, 0.9, 0.2, 0.1].

[0068] For example, if the comparison threshold is 0.5, after comparing each vector value of the violation label prediction vector with the comparison threshold, it can be determined that the target violation categories of the audio to be reviewed are A2 and A3.

[0069] It should be noted that the above examples are merely illustrative. In actual applications, A1, A2, A3, and A4 may be determined based on actual audit requirements and are not limited here. The comparison threshold is also determined based on actual application conditions.

[0070] The audio violation review method provided by the embodiment of the present invention uses a training sample set and an auxiliary data set when training a multi-label classification model. The training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio. The violation mask vector uses different numerical values ​​to represent the actual violation category determined for the training audio and other pending violation categories. The target probability vector includes the probability that the training audio belongs to the actual violation category. In this way, the actual violation category determined for the training audio can be used in combination with the violation mask vector to optimize the multi-label classification model, making the multi-label classification model more accurate. Therefore, in the model application stage, all target violation categories to which the pending audio belongs can be determined, ensuring the comprehensiveness of violation category identification.

[0071] In an optional implementation, the multi-label classification model may adopt a BERT model, a TextCNN model with an adjusted output layer, or a model supporting binary classification with an added multi-output classification layer (eg, a layer of sklearn's Multi Output Classifier).

[0072] The following is an introduction to the training process of the multi-label classification model. Figure 3 The training process may include the following steps S110 to S120:

[0073] S110: Construct a training sample set and an auxiliary data set.

[0074] Optionally, the sub-steps of step S110 may include S111 to S113:

[0075] S111. Obtain several training audios.

[0076] In this embodiment, the training audio may be provided with a violation category label, where the violation category label represents the actual violation category determined for the training audio.

[0077] It is understandable that the actual violation category of the training audio may be the review result obtained by manual review, but the training audio may actually contain other violation categories besides the actual violation category.

[0078] S112: Perform speech recognition on each training audio to obtain a number of ASR texts to form a training sample set.

[0079] S113. For each training audio, determine the violation mask vector and target probability vector of the training audio based on the violation category label of the training audio and multiple preset violation categories, and obtain the violation mask vector and target probability vector of each training audio to form an auxiliary data set.

[0080] In this embodiment, the target probability vector can represent that the probability that the training audio belongs to the actual violation category is 1, and the probabilities of belonging to other pending violation categories are all set parameters. In an optional example, the set parameter can be 0.

[0081] Since the multiple violation categories include audio that does not violate the rules and multiple other violation categories, the violation mask vector includes the vector value corresponding to each violation category. Taking a training audio S as an example, the process of determining the violation mask vector of the training audio based on the violation category label of the training audio and the preset multiple violation categories in step S113 can have two situations:

[0082] First, when the actual violation category is audio non-violation, the vector value corresponding to the actual violation category in the violation mask vector and the vector values ​​corresponding to multiple other violation categories are all set to 1;

[0083] Second, when the actual violation category is one of multiple other violation categories, the vector values ​​corresponding to the actual violation category and the audio non-violation category in the violation mask vector are set to 1, and the vector values ​​corresponding to the pending violation categories other than the actual violation category in the multiple other violation categories are set to 0.

[0084] Therefore, for a training audio, the violation mask vector uses 0 and 1 to represent the uncertain items and confirmed items in multiple violation categories, respectively; the target probability vector uses 1 to represent the probability value of the confirmed items in multiple violation categories, and uses the set parameters to represent the probability value of the uncertain items in multiple violation categories.

[0085] For example, assuming that there are multiple violation categories including A1, A2, A3, A4, and no audio violation, the parameter is set to 0:

[0086] If the actual violation category of training audio S is no violation, and it can be determined that training audio S definitely does not belong to the four violation categories A1, A2, A3, and A4, then the violation mask vector of training audio S is [1, 1, 1, 1, 1]. The target probability vector of training audio S is [0, 0, 0, 0, 1].

[0087] When the actual violation category of training audio S is A1, and it can be determined that training audio S is definitely not a violation-free audio, the violation mask vector of training audio S is [1, 0, 0, 0, 1]. The target probability vector of training audio S is [1, 0, 0, 0, 0].

[0088] It should be noted that this example is only an example and is not limiting.

[0089] S120: Use the training sample set and the auxiliary data set to train the pre-built multi-label classification model to obtain a trained multi-label classification model.

[0090] Optionally, taking one round of training of a multi-label classification model as an example, the sub-steps of step S120 may include S121 to S124:

[0091] S121. Input the training sample set into the multi-label classification model to obtain the prediction probability vector of each ASR text.

[0092] In this embodiment, the predicted probability vector includes a probability value corresponding to each violation category.

[0093] S122. For each ASR text, use the predicted probability vector, target probability vector, and violation Mask vector of the ASR text to calculate the individual loss value of the ASR text.

[0094] Taking the ASR text corresponding to the training audio S as an example, the process of calculating the individual loss value of the ASR text using the prediction probability vector, target probability vector, and violation mask vector of the ASR text in step S122 may include the following steps:

[0095] S122-1. Calculate a pending loss vector based on the target probability vector, the predicted probability vector, and a preset loss function;

[0096] S122-2 multiplies the pending loss vector by the violation mask vector to obtain the target loss vector;

[0097] S122-3. Add each loss value of the target loss vector to obtain the individual loss value.

[0098] In this embodiment, the target probability vector and the predicted probability vector can be substituted into a preset loss function to calculate a pending loss vector. The pending loss vector can include the loss value corresponding to each violation category. In an optional example, the loss function can use Binary Cross Entropy (BCELoss).

[0099] For example, suppose there are five violation categories: A1, A2, A3, A4, and no violation. If the actual violation category of training audio S is A1, then the confirmed violation categories are A1 and no violation, and the uncertain violation categories are A2, A3, and A4. The corresponding violation mask vector of training audio S is [1, 0, 0, 0, 1].

[0100] If the calculated undetermined loss vector for the ASR text corresponding to training audio S is [a, b, c, d, e], then the target loss vector obtained by multiplying the undetermined loss vector by the violation mask vector is [a, 0, 0, 0, e]. Therefore, the individual loss value for the ASR text corresponding to training audio S is a + e. Subsequently, only the loss values ​​for the two deterministic items, A1 and the audio's non-violation status, are used to guide model optimization. The loss values ​​for the uncertain items are not actually included in the individual loss calculation.

[0101] It should be noted that this example is only an example and is not limiting.

[0102] S123. Obtain a target loss value based on the individual loss value of each ASR text.

[0103] In this embodiment, the target loss value may be obtained by adding up the individual loss values ​​of all ASR texts.

[0104] S124. Use the target loss value to perform back propagation to update the model parameters of the multi-label classification model to obtain a trained multi-label classification model.

[0105] As can be seen from the above example, by multiplying the pending loss vector by the violation mask vector, the target loss vector is obtained. Only the loss values ​​of the confirmed items in the various violation categories are non-zero, while the loss values ​​of the uncertain items in the various violation categories are all zero. This ensures that only the loss values ​​of the confirmed items in the various violation categories are used to guide subsequent model optimization. The loss values ​​of the uncertain items in the various violation categories are essentially not included in the loss calculation, which can reduce the impact of erroneous losses and improve the prediction effect when the model is applied.

[0106] It should be noted that the execution order of the steps in the above method embodiment is not limited to that shown in the drawings, and the execution order of the steps shall be based on actual application conditions.

[0107] In order to execute the corresponding steps in the above method embodiment and each possible implementation method, an implementation method of an audio violation review device is given below.

[0108] See Figure 4 , Figure 4 The structure diagram of the apparatus for reviewing audio violations provided by an embodiment of the present invention is shown as follows: The apparatus for reviewing audio violations 200 comprises: an acquisition module 220 and a prediction module 230 .

[0109] The acquisition module 220 is used to acquire the audio to be reviewed and perform speech recognition on the audio to be reviewed to obtain the ASR text to be reviewed;

[0110] Prediction module 230 is configured to: input the ASR text to be reviewed into a trained multi-label classification model to obtain a violation label prediction vector, wherein the violation label prediction vector includes a probability value of the audio to be reviewed belonging to each violation category; and determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector;

[0111] Among them, the multi-label classification model is trained using a training sample set and an auxiliary data set. The training sample set includes the ASR text of several training audios, and the auxiliary data set includes the target probability vector and violation mask vector of each training audio; the violation mask vector uses different numerical values ​​to represent the actual violation category and other pending violation categories of the training audio; the target probability vector includes the probability that the training audio belongs to the actual violation category.

[0112] In an optional embodiment, the audio violation review device 200 may also include a training module 210 for: constructing a training sample set and an auxiliary data set; using the training sample set and the auxiliary data set to train a pre-constructed multi-label classification model to obtain a trained multi-label classification model.

[0113] In an optional embodiment, when the training module 210 is used to construct a training sample set and an auxiliary data set, it can be specifically used to: obtain a number of training audios; the training audios have violation category labels, and the violation category labels represent the actual violation categories that have been determined for the training audios; perform speech recognition on each training audio to obtain a number of ASR texts to constitute a training sample set; for each training audio, determine the violation mask vector and target probability vector of the training audio based on the violation category label of the training audio and a plurality of preset violation categories, and obtain the violation mask vector and target probability vector of each training audio to constitute an auxiliary data set; wherein, the target probability vector represents that the probability that the training audio belongs to the actual violation category is 1, and the probabilities of belonging to other pending violation categories are all set parameters.

[0114] In an optional embodiment, the multiple violation categories include audio non-violation and multiple other violation categories; the violation mask vector includes a vector value corresponding to each violation category. When the training module 210 is used to determine the violation mask vector of the training audio based on the violation category label of the training audio and multiple preset violation categories, it can be specifically configured to: when the actual violation category is audio non-violation, set the vector value corresponding to the actual violation category in the violation mask vector to 1, and set the vector values ​​corresponding to the multiple other violation categories to 0; when the actual violation category is one of the multiple other violation categories, set the vector values ​​corresponding to the actual violation category and audio non-violation in the violation mask vector to 1, and set the vector values ​​corresponding to the undetermined violation categories other than the actual violation category in the multiple other violation categories to 0.

[0115] In an optional embodiment, when the training module 210 is used to train a pre-built multi-label classification model using a training sample set and an auxiliary data set, it can be specifically used to: input the training sample set into the multi-label classification model to obtain a predicted probability vector for each ASR text, the predicted probability vector including the probability value corresponding to each violation category; for each ASR text, using the predicted probability vector, target probability vector and violation Mask vector of the ASR text, calculate the individual loss value of the ASR text; obtain the target loss value based on the individual loss value of each ASR text; and use the target loss value for backpropagation to update the model parameters of the multi-label classification model to obtain a trained multi-label classification model.

[0116] In an optional embodiment, the training module 210 is used to calculate the individual loss value of the ASR text using the predicted probability vector, target probability vector and violation Mask vector of the ASR text. Specifically, it can be used to: calculate the pending loss vector based on the target probability vector, the predicted probability vector and the preset loss function; multiply the pending loss vector by the violation Mask vector to obtain the target loss vector; add each loss value of the target loss vector to obtain the individual loss value.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the audio violation review device 200 described above can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0118] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the audio violation review method disclosed in the above embodiments. The computer-readable storage medium may be, but is not limited to, a USB flash drive, a mobile hard drive, ROM, RAM, PROM, EPROM, EEPROM, a FLASH disk, or an optical disk, among other media capable of storing program code.

[0119] In summary, the embodiments of the present invention provide a method, device, electronic device and readable storage medium for audio violation review. When training a multi-label classification model, a training sample set and an auxiliary data set are used. The training sample set includes ASR texts of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio. The violation mask vector uses different numerical values ​​to represent the actual violation category and other pending violation categories of the training audio. The target probability vector includes the probability that the training audio belongs to the actual violation category. In this way, the target loss value obtained using the violation mask vector during training only retains the influence of the actual violation category, making the multi-label classification model more accurate, so that all target violation categories to which the audio to be reviewed belongs can be determined in the model application stage, ensuring the comprehensiveness of violation category identification.

[0120] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for reviewing audio violations, characterized in that: include: Acquire the audio to be reviewed, and perform speech recognition on the audio to be reviewed to obtain the ASR text to be reviewed; Inputting the ASR text to be reviewed into a trained multi-label classification model to obtain a violation label prediction vector, wherein the violation label prediction vector includes a probability value of the audio to be reviewed belonging to each violation category; Determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector; The trained multi-label classification model is obtained by training in the following way: Constructing a training sample set and an auxiliary data set, wherein the training sample set includes ASR text of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio; the violation mask vector uses different values ​​to represent the actual violation category determined for the training audio and other pending violation categories; the target probability vector includes the probability that the training audio belongs to the actual violation category; Inputting the training sample set into a multi-label classification model to obtain a prediction probability vector for each ASR text, wherein the prediction probability vector includes a probability value corresponding to each violation category; For each ASR text, calculate the individual loss value of the ASR text using the predicted probability vector, target probability vector and violation Mask vector of the ASR text; Get the target loss value based on the individual loss value of each ASR text; Backpropagation is performed using the target loss value to update model parameters of the multi-label classification model to obtain the trained multi-label classification model; The step of calculating the individual loss value of the ASR text using the predicted probability vector, the target probability vector, and the violation Mask vector of the ASR text includes: Calculating a pending loss vector based on the target probability vector, the predicted probability vector, and a preset loss function; Multiplying the pending loss vector by the violation Mask vector to obtain a target loss vector; Each loss value of the target loss vector is added together to obtain the individual loss value.

2. The method according to claim 1, characterized in that The steps of constructing the training sample set and the auxiliary data set include: Acquire a plurality of training audios; the training audios are provided with violation category labels, wherein the violation category labels represent actual violation categories determined for the training audios; Performing speech recognition on each of the training audios to obtain a plurality of ASR texts to constitute the training sample set; For each of the training audios, the violation mask vector and target probability vector of the training audio are determined based on the violation category label of the training audio and multiple preset violation categories, and the violation mask vector and target probability vector of each of the training audios are obtained to constitute the auxiliary data set; wherein, the target probability vector represents that the probability that the training audio belongs to the actual violation category is 1, and the probabilities of belonging to other pending violation categories are all set parameters.

3. The method according to claim 2, characterized in that The multiple violation categories include audio non-violation and multiple other violation categories; the violation Mask vector includes a vector value corresponding to each violation category; The step of determining the violation mask vector of the training audio based on the violation category label of the training audio and multiple preset violation categories includes: When the actual violation category is no audio violation, the vector value corresponding to the actual violation category and the vector values ​​corresponding to multiple other violation categories in the violation Mask vector are all set to 1; When the actual violation category is one of the multiple other violation categories, the vector values ​​corresponding to the actual violation category and the audio non-violation in the violation mask vector are all set to 1, and the vector values ​​corresponding to the pending violation categories other than the actual violation category in the multiple other violation categories are all set to 0.

4. An audio violation review device, characterized in that: include: An acquisition module is used to acquire the audio to be reviewed and perform speech recognition on the audio to be reviewed to obtain the ASR text to be reviewed; A prediction module is configured to: input the ASR text to be reviewed into a trained multi-label classification model to obtain a violation label prediction vector, wherein the violation label prediction vector includes a probability value of the audio to be reviewed belonging to each violation category; and determine at least one target violation category to which the audio to be reviewed belongs based on the violation label prediction vector; The device further includes a training module for: Constructing a training sample set and an auxiliary data set, wherein the training sample set includes ASR text of several training audios, and the auxiliary data set includes a target probability vector and a violation mask vector for each training audio; the violation mask vector uses different values ​​to represent the actual violation category determined for the training audio and other pending violation categories; the target probability vector includes the probability that the training audio belongs to the actual violation category; Inputting the training sample set into a multi-label classification model to obtain a prediction probability vector for each ASR text, wherein the prediction probability vector includes a probability value corresponding to each violation category; For each ASR text, calculate the individual loss value of the ASR text using the predicted probability vector, target probability vector and violation Mask vector of the ASR text; Get the target loss value based on the individual loss value of each ASR text; Backpropagation is performed using the target loss value to update model parameters of the multi-label classification model to obtain the trained multi-label classification model; The training module is specifically used for: Calculating a pending loss vector based on the target probability vector, the predicted probability vector, and a preset loss function; Multiplying the pending loss vector by the violation Mask vector to obtain a target loss vector; Each loss value of the target loss vector is added together to obtain the individual loss value.

5. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-readable instructions to implement the audio violation review method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the audio violation review method as described in any one of claims 1 to 3.

Images