Audio sensitive content detection method, computer device and computer program product

Abstract

The present application relates to a method for detecting audio sensitive content, a computer device and a computer program product. The present application can realize rapid detection of audio content in massive live audio and video, thereby improving the efficiency of audio content detection. The method comprises: obtaining an audio segment to be detected; extracting an audio feature vector to be detected of the audio segment to be detected; inputting the audio feature vector to be detected into a keyword phoneme detection model for detection to obtain a word lattice network; performing a path search on the word lattice network based on the keyword phoneme transfer probability of the keyword phoneme, and obtaining multiple candidate word lattice paths with output probabilities higher than a preset probability threshold; performing speech recognition on the audio segment to be detected corresponding to the multiple candidate word lattice paths to obtain multiple audio text recognition results; performing keyword text detection on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and using the audio segment portion corresponding to the audio text as audio containing sensitive content.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to an audio-sensitive content detection method, computer device, and computer program product. Background Art

[0002] With the development of internet technology, live streaming is becoming increasingly popular. Live streaming can be applied to various industries to promote their development, such as live streaming + e-commerce, live streaming + entertainment, and live streaming + training. Testing the audio content in these applications is conducive to promoting the healthy development of various industries.

[0003] Currently, audio content detection methods primarily include audio keyword detection and speech recognition. Audio keyword detection primarily calculates the acoustic feature similarity between the keyword audio and the audio being tested, while speech recognition detection precisely identifies the audio being tested as text and then detects whether it contains the preset keyword text. However, in real-time, high-volume audio scenarios, these technologies suffer from low audio content detection efficiency. Summary of the Invention

[0004] Based on this, it is necessary to provide an audio sensitive content detection method, computer device and computer program product to address the above technical issues.

[0005] In a first aspect, the present application provides a method for detecting audio-sensitive content. The method comprises:

[0006] Get the audio clip to be detected;

[0007] Extracting an audio feature vector to be detected from the audio segment to be detected;

[0008] Inputting the audio feature vector to be detected into a trained keyword phoneme detection model for detection to obtain a word lattice network;

[0009] Based on the keyword phoneme transition probability of the keyword phoneme, a path search is performed on the word lattice network to obtain a plurality of candidate word lattice paths whose output probability is higher than a preset probability threshold; wherein the keyword phoneme is obtained by decomposing a preset keyword;

[0010] Performing speech recognition on the audio segments to be detected corresponding to the multiple candidate word pattern paths to obtain multiple audio text recognition results;

[0011] Keyword text detection is performed on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and the audio segment portion corresponding to the audio text in the audio segment to be detected is used as audio containing sensitive content.

[0012] In a second aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above-mentioned audio-sensitive content detection method embodiment when executing the computer program.

[0013] In a third aspect, the present application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in the above-mentioned audio-sensitive content detection method embodiment.

[0014] The above-mentioned audio sensitive content detection method, computer equipment and computer program product obtain the audio segment to be detected; extract the audio feature vector to be detected of the audio segment to be detected; input the audio feature vector to be detected into the trained keyword phoneme detection model for detection to obtain a word lattice network; based on the keyword phoneme transfer probability of the keyword phoneme, perform path search on the word lattice network to obtain multiple candidate word lattice paths with output probabilities higher than a preset probability threshold; wherein the keyword phoneme is obtained by decomposing a preset keyword; perform speech recognition on the audio segment to be detected corresponding to the multiple candidate word lattice paths to obtain multiple audio text recognition results; perform keyword text detection on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and use the audio segment portion corresponding to the audio text in the audio segment to be detected as audio containing sensitive content. The present application detects audio by a multi-model cascade method, which can realize accurate and rapid detection of audio content in massive live audio and video, improve the efficiency of audio content detection, ensure the timeliness and accuracy of audio content detection, reduce false recalls, and save manual review costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A diagram illustrating an application environment of an audio-sensitive content detection method according to an embodiment;

[0016] Figure 2 1 is a flow chart of a method for detecting audio-sensitive content in one embodiment;

[0017] Figure 3 2 is a flow chart of a method for detecting audio-sensitive content in another embodiment;

[0018] Figure 4 Schematic diagram of a flow chart of a keyword detection step in another embodiment;

[0019] Figure 5 1 is a flow chart of a method for detecting audio-sensitive content in one embodiment;

[0020] Figure 6 1 is a schematic diagram of a calculation flow of a keyword phoneme transition probability in one embodiment;

[0021] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0023] The audio sensitive content detection method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 101 communicates with the server 102 through the network. The data storage system can store the data that the server 102 needs to process. The data storage system can be integrated on the server 102, or it can be placed on the cloud or other network servers. Among them, the terminal 101 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 102 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0024] In one embodiment, Figure 2 As shown, a method for detecting audio sensitive content is provided, which is applied to Figure 1 Taking the server 102 in the example as an example, the following steps are included:

[0025] Step S201: Acquire an audio segment to be detected.

[0026] The audio segment to be detected is a valid audio segment obtained by preprocessing the original audio signal. The original audio signal refers to the original video, such as a live audio data stream or a live video data stream. Preprocessing can be performed using preprocessing techniques such as voice endpoint detection. This preprocessing removes silence or noise from the original audio signal to obtain a valid audio segment that is easy to process.

[0027] Specifically, the original voice signal, including audio or video streams, is obtained through the background of the live broadcast software. Since there are many publishers of live broadcast works, audio and video streams can be obtained in batches. For example, the live broadcast stream link reported by the publisher of the work can be received in the background of the live broadcast software, and the live stream can be downloaded or pulled in real time. In order to improve the efficiency of audio detection, it is also necessary to extract the audio channel signal from the live stream, perform unified coding format conversion, normalize and pre-emphasize the audio, and obtain a valid voice segment, that is, the above-mentioned audio segment to be detected.

[0028] Step S202: extracting an audio feature vector of the audio segment to be detected;

[0029] The audio feature vector to be detected is a computer language description of the audio segment to be detected. Since the physical waveform of sound cannot be directly recognized by the computer, the waveform needs to be transformed to obtain a digital signal that can describe the speech characteristics.

[0030] Specifically, it is necessary to extract the audio feature vector to be detected in the audio segment to be detected ,in, Represents the audio feature vector of the tth frame; specifically, the audio segment to be detected is framed, that is, the audio segment to be detected is cut into small segments, each of which is called a frame, and each frame is approximately 10ms. After framing, the speech is divided into many small segments, that is, multi-frame speech signals. However, the waveform has almost no descriptive ability in the time domain, so the waveform must be transformed. The transformation method used in this embodiment is to extract acoustic features through short-time Fourier transform (STFT, short-time Fourier transform, or short-term Fourier transform) to obtain the audio feature vector to be detected. The audio feature vector to be detected can be described by one or more of Mel-log cepstrum, Fbank (FilterBank), and MFCC (Mel Frequency Cepstral Coefficent).

[0031] Step S203: inputting the audio feature vector to be detected into a trained keyword phoneme detection model for detection to obtain a word lattice network;

[0032] The keyword phoneme detection model is a speech recognition neural network model that can identify phonemes similar to those in the audio stream. Lattice networks are a representation used in speech recognition to represent recognition results.

[0033] like Figure 3 As shown, Figure 3 This is another flowchart of the above audio sensitive content detection method; the audio feature vector to be detected Input the trained keyword phoneme detection model to generate a word lattice network corresponding to the audio clip to be detected. The word lattice network contains multiple phonemes to be detected and the weights corresponding to each phoneme to be detected, such as (wo, 0.5), (shi, 0.3), (shui, 0.2).

[0034] Step S204: Based on the keyword phoneme transfer probability of keyword phonemes, perform path search on the lattice network to obtain multiple candidate lattice paths with output probabilities higher than a preset probability threshold; wherein, the keyword phonemes are obtained by decomposing a preset keyword.

[0035] Among them, the preset keyword refers to a sensitive word set by humans. A phoneme is the basic unit that constitutes sound, and the pronunciation of a word is composed of phonemes. For example, in English, a phoneme set consisting of 39 phonemes can be used, and in Chinese, all initials and finals can be directly used as the phoneme set. One or more phonemes combine to form a word. Therefore, the above-mentioned preset keyword is composed of specific keyword phonemes. For example, the phonemes that make up the word "Xiaoming" are "x", "iao", "m", and "ing". For the keyword phoneme transfer probability, in a speech recognition model, an HMM (Hidden Markov Model) is commonly used to construct the speech recognition model. The model includes a chain structure composed of multiple states. In the chain structure, the transfer of internal states is related to the previous state. The keyword phoneme transfer probability refers to the transfer probability between two phonemes. For example, in the above "Xiaoming", a certain probability can be statistically obtained according to Chinese grammar rules from "x" to "iao", and this probability is called the keyword phoneme transfer probability. The output probability refers to the probability of the finally obtained character. For example, the probability of obtaining "xiao" as mentioned above is the output probability.

[0036] Specifically, as Figure 3 shown, input the lattice network detected above into a trained classifier, so that the classifier performs path search in the above lattice network based on the keyword phoneme transfer probability of the preset keyword phonemes, and obtains multiple candidate lattice paths with output probabilities higher than the preset probability threshold. For example, if the preset keyword is "Xiaoming", then search for lattice paths (phoneme paths) similar to "Xiaoming" in the above lattice network based on the keyword phoneme transfer probability of "Xiaoming", and obtain multiple candidate lyric paths (N best lattices). For example, it may obtain "xiao’ming" (Xiaoming) or "xian’ming" (distinct) etc.

[0037] Step S205: Perform speech recognition on the audio segments to be detected corresponding to multiple candidate lattice paths to obtain multiple audio text recognition results.

[0038] Among them, speech recognition is a general speech recognition model, that is, a model that recognizes speech as text.

[0039] Specifically, as Figure 3As shown, the audio segments to be detected corresponding to the above multiple candidate lattices paths (N best lattices) are input into the speech recognition model for speech recognition, and the text results corresponding to each candidate lattice path are obtained, such as the above "Xiao Ming" and "Xian Xian".

[0040] Step S206: perform keyword text detection on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and use the audio segment portion corresponding to the audio text in the audio segment to be detected as the audio containing sensitive content.

[0041] Specifically, after recognizing the above text results, although humans can clearly tell which ones are keyword hits and which ones are not, for machines (computers), computers cannot understand the inner meaning of these texts, so they cannot know which ones are actually keyword hits. Therefore, it is necessary to perform keyword text detection on the above different text detection results, such as Figure 3 As shown, text detection is performed on the different text results to detect results that match the text of the preset keyword. The audio clip corresponding to this result is then identified as audio containing sensitive content. The text recognition results are input into a trained character recognition model, such as a CTC (Connectionist temporal classification) model, to obtain the characters corresponding to the suspected hit audio output by the character recognition model. Keyword matching is then performed on the characters, that is, the output text is then subjected to keyword matching to determine whether it matches the preset keyword. If the keyword is hit and the probability score exceeds a set threshold, the suspected hit audio is recalled and sent to the manual review channel. For example, if the text "Xiao Ming" and "Xingming" are detected, the text that matches the preset keyword text "Xiao Ming" is obtained, thereby finding the corresponding audio clip to be detected. This clip is identified as audio containing sensitive content and recalled for manual review.

[0042] The above embodiment obtains an audio segment to be detected; extracts an audio feature vector to be detected of the audio segment to be detected; inputs the audio feature vector to be detected into a trained keyword phoneme detection model for detection to obtain a word lattice network; based on the keyword phoneme transfer probability of the keyword phoneme, performs a path search on the word lattice network to obtain multiple candidate word lattice paths with output probabilities higher than a preset probability threshold; wherein the keyword phoneme is obtained by decomposing a preset keyword; performs speech recognition on the audio segment to be detected corresponding to the multiple candidate word lattice paths to obtain multiple audio text recognition results; performs keyword text detection on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and uses the audio segment portion corresponding to the audio text in the audio segment to be detected as audio containing sensitive content. This embodiment detects audio content through a cascade of multiple models such as a keyword phoneme detection model, a speech recognition model, and a text detection model. It can achieve accurate and rapid detection of audio content in massive live audio and video. Compared with traditional full-scale speech recognition, this embodiment only needs to perform speech recognition and text detection on a small number of candidate word paths that hit keyword phonemes, which greatly reduces the background calculation amount, saves background computing resources, improves the efficiency of audio content detection, ensures the timeliness and accuracy of audio content detection, reduces false recalls, and saves manual review costs.

[0043] In one embodiment, the above-mentioned step S204 includes: calculating the output probability of each word lattice path in the word lattice network based on the keyword phoneme transfer probability of the keyword phoneme; inputting the output probability of each word lattice path into the classifier to obtain multiple candidate word lattice paths that are higher than a preset probability threshold.

[0044] Specifically, if Figure 3As shown, input the lattice network obtained from the above detection into the trained classifier, so that the classifier performs path search in the above lattice network based on the keyword phoneme transition probability of the preset keyword phoneme. Since the lattice network contains the probabilities of each phoneme, and these phonemes are converted into corresponding characters, the probability of the output character (i.e., the output probability) is directly related to the phoneme probability (transition probability) that makes up the character. In order to detect whether the audio segment contains the preset keyword, it is first necessary to calculate the output probability of each lattice path in the above lattice network, and then calculate multiple candidate lattice paths with an output probability higher than the preset probability threshold. For example, if the preset keyword is "Xiaoming", then based on the keyword phoneme transition probability of "Xiaoming", search for lattice paths (phoneme paths) similar to "Xiaoming" in the above lattice network to obtain multiple candidate lyric paths (N best lattices). For example, it may obtain "xiao’ming" (Xiaoming) with an output probability of 0.8, or "xian’ming" (distinct) with an output probability of 0.2; input these output probabilities into the classifier to obtain an output probability higher than the preset probability threshold (for example, 0.7), such as obtaining "xiao’ming" (Xiaoming) with the above output probability of 0.8. Further, at this time, the audio segment corresponding to "xiao’ming" (Xiaoming) with an output probability of 0.8 can be recalled as a suspected hit audio, and the audio segment with a probability lower than the preset probability threshold can be used as normal audio content.

[0045] In the above embodiment, by setting a classifier to filter out lattice paths with a probability lower than the preset probability threshold, obtain lattice paths with a probability higher than the preset probability threshold, and recall the audio corresponding to the lattice paths with a probability higher than the preset probability threshold as suspected hit audio for further detection in the background. This step can filter out most normal audio content and avoid pulling normal audio content into the background for recognition detection, wasting too much computing resources. [[ID=​​​​​​​​​​​​​The first encoder is a deep neural network encoder with a Conformer (heterogeneous, or hybrid) structure, which is used to calculate the high-level acoustic representation of the acoustic features. In this application, it is used to calculate the high-level vector of the above-mentioned audio feature vector to be detected. The first encoder can also be other deep neural network structures, including but not limited to CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Transformer, etc. The above-mentioned audio feature vector to be detected is processed by the first encoder to obtain a high-level embedded audio feature. ,in, is the high-level embedded audio feature of the t-th frame, such as Figure 5 As shown in .

[0050] Optionally, the first encoder mainly introduces more convolutional layers into the model to improve feature expression capabilities. The first encoder can also be other types of deep neural network structures, such as CNN (Convolutional Neural Networks), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory), Transformer, etc.

[0051] In step S402 , the keyword phonemes of the preset keywords are calculated based on the attention mechanism to obtain the keyword phoneme transfer probability of each keyword phoneme.

[0052] Among them, the keyword phoneme transfer probability refers to the transfer probability between the keyword phonemes that constitute the preset keyword, such as Figure 5 in As shown. The attention mechanism can focus on a focal part (giving it more weight) at a specific moment and ignore other parts. In this embodiment, the attention mechanism can be used to calculate the keyword phoneme transition probability of the preset keyword phoneme.

[0053] Specifically, the keyword phonemes of the preset keywords are calculated through the attention mechanism to obtain the keyword phoneme transfer probability of each keyword phoneme , represents the transition probability of the u-th keyword phoneme.

[0054] Step S403 : decoding and identifying the high-level embedded audio features of the audio segment to be detected by combining the keyword phoneme transition probabilities of the keyword phonemes through a joint network to obtain a word lattice network corresponding to the audio segment to be detected.

[0055] The joint network is a decoder network, which is mainly a deep neural network composed of some feedforward layers. The joint network can fuse the output of the first encoder and the output of the preset keyword encoding. Represents the mapping relationship between the audio features of the t-th frame and the u-th phoneme.

[0056] Specifically, the above high-level embedding audio features And the above keyword phoneme transfer probability vector Input the joint network to obtain the word grid network corresponding to the audio segment to be detected ,in, It represents the mapping relationship between the high-level embedded audio features of the t-th frame and the u-th phoneme. These mapping relationships can be represented by weights, where the mapping relationship related to the preset keyword phoneme is given a higher weight.

[0057] In the above embodiment, the keyword detection model is combined with preset keywords to decode and identify the above high-level embedded audio features to obtain a word lattice network of the audio segment to be detected, providing a data basis for subsequent classifier classification.

[0058] In one embodiment, the trained keyword phoneme detection model further includes a second encoder, an attention encoding module, and a prediction network; Figure 6 As shown, the above step S402 includes:

[0059] Step S601: input multiple keyword phonemes corresponding to the preset keywords into a second encoder to obtain keyword phoneme embedding features output by the second encoder;

[0060] Specifically, if Figure 5 As shown, the preset keyword (Key) is converted into a keyword phoneme sequence .in, The second encoder uses the keyword offset algorithm to extract the keyword phoneme sequence embedding features of the keyword phoneme sequence Y. .

[0061] Step S602: embed keyword phonemes into features Input the attention encoding module, so that the attention encoding module encodes based on the time series relationship between keyword phonemes to obtain the context feature vector of each keyword phoneme ;

[0062] Among them, the context feature vector It refers to the features obtained based on the time series relationship of the keyword phonemes. For example, in "Xiaoming", after detecting "iao", the probability of getting "m" is the highest, and other phonemes are unlikely. Therefore, the context feature vector of the keyword phoneme of the preset keyword can be obtained based on these time or position sequence relationships. The attention encoding module can focus on the positional relationship of keyword phonemes and output a contextual feature vector that reflects the relationship between keyword phoneme sequences. .

[0063] Specifically, keyword phonemes are embedded into features Input attention encoding module, the attention encoding module can encode based on the time series relationship between the keyword phonemes to obtain the context feature vector of each keyword phoneme .

[0064] Step S603: taking each keyword phoneme in the plurality of keyword phonemes as the current keyword phoneme, and predicting the context feature vector of the current keyword phoneme through the prediction network. And the preceding phoneme sequence of the current keyword phoneme, predict the keyword phoneme transfer probability of the current keyword phoneme ;

[0065] Specifically, the prediction network is a phoneme-based language model that calculates the transition probability between keyword phoneme labels and inputs the first u-1 phonemes. And the phoneme context feature vector output by the above attention model that reflects the context information of the keyword phoneme sequence , after the prediction network outputs the keyword phoneme transfer probability vector of the u-th phoneme The keyword phoneme transition probability vector Depends on the phoneme context feature vector of the above preset keywords And the preceding keyword phoneme (i.e. the first u-1 keyword phonemes The prediction network learns the association of phoneme context information through a deep neural network (such deep neural networks include but are not limited to RNN, LSTM, Transformer, BERT, etc.) or an N-gram model to predict the current output phoneme label. At the same time, because the input includes the output of the above-mentioned attention model, the output of the prediction network is biased towards the phoneme recognition result of the preset keyword.

[0066] The above embodiment, by using the second encoder and the attention model that utilizes the attention mechanism, can increase the attention weight of preset keywords, guide the matching of preset keywords more favorably during keyword prediction and search, and improve the accuracy and real-time performance of keyword detection.

[0067] Furthermore, the above embodiment supports rapid model updating when keywords are flexibly added or deleted without retraining the model, which simplifies the process of audio content detection and reduces model complexity.

[0068] In one embodiment, the above step S602 includes: taking each keyword phoneme in the plurality of keyword phonemes as the current keyword phoneme, and for the current keyword phoneme, using the attention encoding module based on the hidden layer node state of the preceding phoneme sequence of the current keyword phoneme and the keyword phoneme embedding features of the current keyword phoneme , encode to obtain the context feature vector of the current keyword phoneme.

[0069] Specifically, if Figure 5 As shown, for each keyword phoneme, its context feature vector is calculated. For example, for the u-th keyword phoneme , through the attention encoding module based on the hidden layer node state of its previous phoneme sequence (i.e., the first u-1 phoneme sequence) ( ) and the current keyword phoneme Keyword phoneme embedding features , encode the current keyword phoneme The context feature vector .

[0070] In the above embodiment, the context feature vector of each keyword phoneme is calculated through the attention encoding module, providing a data basis for the subsequent identification of the word lattice network of the audio segment to be detected.

[0071] In one embodiment, the above-mentioned step S205 includes: inputting the audio segments to be detected corresponding to multiple candidate word paths into the speech recognition model, so that the speech recognition model decodes the audio segments to be detected based on the word sequence relationship to obtain multiple audio text recognition results.

[0072] Specifically, the audio segments to be detected corresponding to the multiple candidate lattice paths (N best lattices) are input into the speech recognition model for decoding based on the word sequence relationship to obtain multiple audio text recognition results.

[0073] In the above embodiment, multiple texts are recognized by the speech recognition model, which is conducive to the subsequent detection of texts that are consistent with the keyword text.

[0074] In one embodiment, the above step S201 includes: obtaining an original speech signal; performing audio endpoint detection on the original speech signal to obtain an audio segment to be detected.

[0075] Specifically, the original speech signal is first subjected to audio endpoint detection. Audio endpoint detection, also known as voice boundary detection or VAD, involves removing silence or background noise at the beginning and end of the speech audio before speech recognition begins to reduce interference with subsequent steps. The resulting audio segment is called the audio segment to be detected. Specifically, the amplitude, energy, zero-crossing rate, and fundamental frequency of the original speech signal are detected to remove silent and noisy segments, extract valid speech segments, and minimize their impact on the recognition results.

[0076] The above embodiment can make subsequent speech recognition more accurate and faster through audio endpoint detection.

[0077] In one embodiment, the above method also includes: inputting the output probability of each word lattice path into a classifier to obtain a word lattice path that is lower than or equal to a preset probability threshold; and treating the audio segment to be detected corresponding to the word lattice path that is lower than or equal to the preset probability threshold as a normal audio segment.

[0078] Among them, the classifier is a classifier of the blanking model, which can determine whether the acoustic feature vector of the audio frame to be detected hits the keyword and output its probability to obtain the keyword recognition result. If the keyword is hit and the probability exceeds the preset threshold, the audio frame will be treated as a suspected hit audio for the next step of processing.

[0079] Specifically, the classifier can embed high-level audio features And the above keyword phoneme transfer probability vector The mapping relationship between them is classified, and the audio feature vectors that hit the preset keywords are distinguished by the preset threshold, and the corresponding audio clips are regarded as suspected hit audios, while the audio clips below the preset threshold are regarded as normal audio content.

[0080] The above embodiment reduces the workload of background detection and saves computing power by detecting normal audio content and releasing the normal audio content.

[0081] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0082] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as preset keywords and audio-sensitive content detection results. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an audio-sensitive content detection method is implemented.

[0083] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0084] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above-mentioned audio-sensitive content detection method embodiment when executing the computer program.

[0085] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above-mentioned audio-sensitive content detection method embodiment when executed by a processor.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0087] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0088] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for detecting audio-sensitive content, characterized in that: The method comprises: Get the audio clip to be detected; Extracting an audio feature vector to be detected from the audio segment to be detected; Obtaining high-level embedded audio features of the audio segment to be detected through a first encoder in a trained keyword phoneme detection model; calculating keyword phonemes of preset keywords based on an attention mechanism to obtain keyword phoneme transition probabilities of each keyword phoneme; Decoding and identifying the high-level embedded audio features of the audio segment to be detected by combining the keyword-phoneme transition probabilities of each keyword-phoneme through the joint network of the keyword-phoneme detection model to obtain a word lattice network corresponding to the audio segment to be detected; the word lattice network includes a plurality of phonemes to be detected that are similar to the phonemes constituting the preset keyword and the probabilities of each of the phonemes to be detected; Based on the keyword phoneme transition probability of the keyword phoneme, a path search is performed on the word lattice network to obtain a plurality of candidate word lattice paths whose output probability is higher than a preset probability threshold; wherein the keyword phoneme is obtained by decomposing a preset keyword; Performing speech recognition on the audio segments to be detected corresponding to the multiple candidate word pattern paths to obtain multiple audio text recognition results; Keyword text detection is performed on the multiple audio text recognition results to obtain audio text that hits the preset keyword text; and the audio segment portion corresponding to the audio text in the audio segment to be detected is used as audio containing sensitive content.

2. The method according to claim 1, characterized in that The keyword phoneme transfer probability based on the keyword phoneme is used to search for a path in the word lattice network to obtain a plurality of candidate word lattice paths whose output probabilities are higher than a preset probability threshold, including: Calculating the output probability of each word lattice path in the word lattice network based on the keyword phoneme transition probability of the keyword phoneme; The output probabilities of the respective word lattice paths are input into a classifier to obtain the plurality of candidate word lattice paths that are higher than the preset probability threshold.

3. The method according to claim 1, characterized in that The first encoder comprises a deep neural network encoder, and the joint network comprises a deep network consisting of a plurality of feed-forward layers.

4. The method according to claim 1, wherein The trained keyword phoneme detection model further includes a second encoder, an attention encoding module, and a prediction network; the keyword phonemes of the preset keywords are calculated based on the attention mechanism to obtain the keyword phoneme transition probability of each keyword phoneme, including: Inputting a plurality of keyword phonemes corresponding to the preset keyword into the second encoder to obtain keyword phoneme embedding features output by the second encoder; Inputting the keyword phoneme embedding features into the attention encoding module, so that the attention encoding module encodes based on the time series relationship between the keyword phonemes to obtain a context feature vector for each keyword phoneme; Each keyword phoneme among the multiple keyword phonemes is taken as the current keyword phoneme, and for the current keyword phoneme, the keyword phoneme transition probability of the current keyword phoneme is predicted based on the context feature vector of the current keyword phoneme and the preceding phoneme sequence of the current keyword phoneme by the prediction network.

5. The method according to claim 4, characterized in that The keyword phoneme embedding feature is input into the attention encoding module, so that the attention encoding module encodes based on the time series relationship between the keyword phonemes to obtain a context feature vector for each keyword phoneme, including: Each keyword phoneme among the multiple keyword phonemes is taken as the current keyword phoneme. For the current keyword phoneme, the attention encoding module is used to encode the context feature vector of the current keyword phoneme based on the hidden layer node state of the preceding phoneme sequence of the current keyword phoneme and the keyword phoneme embedding feature of the current keyword phoneme.

6. The method according to claim 1, characterized in that The performing speech recognition on the audio segments to be detected corresponding to the multiple candidate word pattern paths to obtain multiple audio text recognition results includes: The audio segments to be detected corresponding to the multiple candidate word lattice paths are input into a speech recognition model, so that the speech recognition model decodes the audio segments to be detected based on the word sequence relationship to obtain the multiple audio text recognition results.

7. The method according to claim 1, characterized in that The step of obtaining the audio segment to be detected includes: Obtaining the original speech signal; Audio endpoint detection is performed on the original speech signal to obtain the audio segment to be detected.

8. The method according to claim 2, characterized in that The method further comprises: Inputting the output probabilities of the respective word lattice paths into the classifier to obtain word lattice paths that are lower than or equal to the preset probability threshold; The audio segment to be detected corresponding to the word lattice path that is lower than or equal to the preset probability threshold is regarded as a normal audio segment.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images