Audio detection method, device, computer device, storage medium

Abstract

This application relates to an audio detection method, apparatus, computer device, storage medium, and computer program product. The method includes: obtaining audio data; performing segmentation processing on the audio data to obtain a plurality of audio sequences; respectively detecting the plurality of audio sequences; if there is abnormal audio data in the audio sequence, obtaining an abnormal score for the audio sequence with the abnormal audio data; and outputting an audio detection report according to the abnormal score. Using this method can improve the accuracy of audio detection.

Description

Technical Field

[0001] This application relates to the field of computer applications, and in particular, to an audio detection method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] Since sound information can be used in scenarios with poor lighting, visual blind spots, etc., audio anomaly detection technology is an important supplement to existing video surveillance systems. Due to the scarcity of abnormal sounds that occur in application scenarios, traditional binary classification models are prone to missed detections during detection. Summary of the Invention

[0003] Based on this, it is necessary to provide an audio detection method, apparatus, computer device, computer-readable storage medium, and computer program product for the above technical problems.

[0004] In a first aspect, this application provides an audio detection method. The method includes:

[0005] Obtain audio data;

[0006] Perform segmentation processing on the audio data to obtain a plurality of audio sequences;

[0007] Detect each of the plurality of audio sequences respectively;

[0008] If there is abnormal audio data in the audio sequence, obtain an anomaly score for the audio sequence with the abnormal audio data;

[0009] Output an audio detection report according to the anomaly score.

[0010] In one embodiment, the performing segmentation processing on the audio data to obtain a plurality of audio sequences includes:

[0011] Perform frame division processing on the audio data according to a preset frame length to obtain a plurality of audio sequences with the same frame length.

[0012] In one embodiment, the detecting each of the plurality of audio sequences respectively includes:

[0013] Calculate the spectral energy of the audio sequence;

[0014] Detect the plurality of audio sequences through the spectral energy.

[0015] In one embodiment, the obtaining an anomaly score for the audio sequence with the abnormal audio data if there is abnormal audio data in the audio sequence includes:

[0016] Performing feature extraction on the audio sequence containing the abnormal audio data to obtain a first acoustic feature of the audio sequence containing the abnormal audio data;

[0017] A first detection model is retrieved, and the first acoustic feature is input into the first detection model to obtain an anomaly score.

[0018] In one embodiment, calling the first detection model and inputting the first acoustic feature into the first detection model to obtain an anomaly score includes:

[0019] Calculating a first similarity between the first acoustic feature and an acoustic feature of abnormal audio data in a memory;

[0020] Inputting the first acoustic feature into a preset anomaly detector to calculate an anomaly degree score;

[0021] An anomaly score is calculated using the first similarity and the anomaly degree score.

[0022] In one embodiment, the method further includes:

[0023] If no abnormal audio data exists in the audio sequence, extracting acoustic features from the multiple audio sequences to obtain a second acoustic feature corresponding to each audio sequence in the multiple audio sequences;

[0024] Inputting the second acoustic feature into a second detection model, and calculating a second similarity between the multiple audio sequences;

[0025] An anomaly score is calculated based on the second similarity.

[0026] In a second aspect, the present application further provides an audio detection device. The device comprises:

[0027] A data acquisition module, used to acquire audio data;

[0028] A processing module, configured to segment the audio data to obtain a plurality of audio sequences;

[0029] A detection module, configured to detect each of the plurality of audio sequences;

[0030] a score obtaining module, configured to obtain an anomaly score for the audio sequence containing abnormal audio data if abnormal audio data exists in the audio sequence;

[0031] An output module is used to output an audio detection report based on the anomaly score.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0033] Obtain audio data;

[0034] Perform segmentation processing on the audio data to obtain a plurality of audio sequences;

[0035] Detect each of the plurality of audio sequences separately;

[0036] If there is abnormal audio data in the audio sequence, obtain an abnormal score for the audio sequence with the abnormal audio data;

[0037] Output an audio detection report according to the abnormal score.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtain audio data;

[0040] Perform segmentation processing on the audio data to obtain a plurality of audio sequences;

[0041] Detect each of the plurality of audio sequences separately;

[0042] If there is abnormal audio data in the audio sequence, obtain an abnormal score for the audio sequence with the abnormal audio data;

[0043] Output an audio detection report according to the abnormal score.

[0044] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Obtain audio data;

[0046] Perform segmentation processing on the audio data to obtain a plurality of audio sequences;

[0047] Detect each of the plurality of audio sequences separately;

[0048] If there is abnormal audio data in the audio sequence, obtain an abnormal score for the audio sequence with the abnormal audio data;

[0049] Output an audio detection report according to the abnormal score.

[0050] The above audio detection method, device, computer device, storage medium, and computer program product obtain audio data; segment the audio data to obtain multiple audio sequences; detect the multiple audio sequences respectively; if there is abnormal audio data in the audio sequence, obtain an abnormal score for the audio sequence with the abnormal audio data; and output an audio detection report according to the abnormal score. By obtaining audio data, segmenting the audio data to obtain multiple audio sequences, detecting whether there is abnormal audio data in the audio sequences, obtaining an abnormal score for the audio sequences with abnormal audio data, and then outputting an audio detection report, a comprehensive detection of the audio can be implemented. Description of the Drawings

[0051] Figure 1 It is an application environment diagram of the audio detection method in an embodiment;

[0052] Figure 2 It is a schematic flowchart of the audio detection method in an embodiment;

[0053] Figure 3 It is a schematic diagram of the process of obtaining an abnormal score by the memory detection model in the audio detection method in an embodiment;

[0054] Figure 4 It is a schematic diagram of the process of obtaining an abnormal score by the contrast detection model in the audio detection method in an embodiment;

[0055] Figure 5 It is a schematic flowchart of the audio detection method in another embodiment;

[0056] Figure 6 It is a schematic diagram of the feature extraction process in the audio detection method in an embodiment;

[0057] Figure 7 It is a structural block diagram of the audio detection device in an embodiment;

[0058] Figure 8 It is an internal structure diagram of a computer device in an embodiment. Detailed Embodiments

[0059] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] The audio detection method provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed on the cloud or other network servers. The server 104 obtains the audio data of the terminal 102; segments the audio data to obtain multiple audio sequences; detects each of the multiple audio sequences; if there is abnormal audio data in the audio sequence, calculates an abnormal score for the audio sequence with abnormal audio data; outputs an audio detection report according to the abnormal score. Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0061] It can be understood that the voice processing method provided in the embodiments of the present application can be applied not only to a system including a terminal and a server, but also can be applied alone to a terminal or a server.

[0062] In one embodiment, as Figure 2 shown, an audio detection method is provided. Taking the method applied to Figure 1 the server 104 or the terminal 102 in

[0063] Step 202, obtain audio data.

[0064] Among them, the audio data can be a sound signal collected in real time, or a sound signal stored in a storage space after collection. The user can directly obtain the audio signal from a sound collection device, or can separate the audio data from other mixed signals containing audio such as video-audio signals.

[0065] In a possible implementation manner, the processor of the terminal obtains the audio data collected by the sound collection device.

[0066] In another possible implementation manner, the terminal uploads the audio data collected by the sound collection device to the server.

[0067] Step 204, segment the audio data to obtain multiple audio sequences.

[0068] Among them, the audio sequence refers to the audio segment obtained by splitting audio data into tags. The segmentation process of the audio is to split the complete audio data into several small audio segments, and each segment is independent of each other. The audio segmentation methods include: distance-based audio segmentation method, model selection-based audio segmentation method, model-based audio segmentation method, and hybrid audio segmentation method, etc. In the distance-based audio segmentation method, a distance scale is defined to evaluate the similarity between two adjacent windows, thereby forming a distance curve. According to the set threshold, the peak position of the distance curve is selected as a change point. It is assumed that the feature vectors in adjacent windows follow a certain probability density, and the distance is the difference between the two probability densities. Common distance scales include KL distance (Kullback-Lcibler Distance), GLR distance (Generalized Likelihood Ratio), Mahalanobis Distance, and Bhattacharyya Distance. In the model selection-based audio segmentation method, the commonly used one is the audio segmentation method based on the Bayesian Information Criterion (BIC), which simulates the problem of finding the jump point as a problem of selecting between two models. In the model-based audio segmentation method, different audio types are modeled, and then each frame of audio is classified using maximum likelihood estimation. The hybrid audio segmentation method is usually a new hybrid algorithm obtained by combining the distance-based audio segmentation method, the model selection-based audio segmentation method, and the model-based audio segmentation method. The CLDNNs structure is used to solve the Voice Activity Detection (VAD) problem, which combines the Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Deep Neural Network (DNN) for modeling.

[0069] Specifically, the processor splits the audio data into multiple audio sequences according to a preset audio segmentation method. The preset audio segmentation method may include any one of the distance-based audio segmentation method, the model selection-based audio segmentation method, the model-based audio segmentation method, and the hybrid audio segmentation method, etc.

[0070] Step 206, detect each of the multiple audio sequences separately.

[0071] Among them, detecting the audio sequence refers to detecting whether there is abnormal audio data in the audio sequence.

[0072] Specifically, the processor separately detects each of the multiple audio sequences for the existence of abnormal audio data.

[0073] Step 208, if there is abnormal audio data in the audio sequence, calculate the anomaly score for the audio sequence with abnormal audio data.

[0074] Among them, abnormal audio data refers to audio information in the audio that is easily confused and has a negative impact. For example, in an actual meeting scenario, since the sound of typing on the keyboard and the sound of closing the door are easily recognized as gunshots, footsteps are also easily confused with gunshots, and coughing sounds and screaming sounds have a certain similarity, data related to public security threat events such as explosions, gunshots, and screaming sounds belong to abnormal audio data. The anomaly score represents the degree of abnormality of the audio sequence with abnormal audio data. The method of calculating the anomaly score for the audio sequence with abnormal audio data is generally obtained through a model.

[0075] Specifically, if there is abnormal audio data in the audio sequence, the processor calculates the anomaly score for the audio sequence with abnormal audio data.

[0076] Step 210, output an audio detection report according to the anomaly score.

[0077] Among them, the audio detection report includes the anomaly score, anomaly level, etc. of the audio sequence with abnormal audio data.

[0078] Specifically, the processor outputs an audio detection report according to the anomaly score.

[0079] In the above audio detection method, obtain audio data; segment the audio data to obtain multiple audio sequences; detect each of the multiple audio sequences; if there is abnormal audio data in the audio sequence, calculate the anomaly score for the audio sequence with abnormal audio data; output an audio detection report according to the anomaly score. By obtaining audio data, segmenting the audio data to obtain multiple audio sequences, detecting whether there is abnormal audio data in the audio sequences, calculating the anomaly score for the audio sequences with abnormal audio data, and then outputting an audio detection report, comprehensive detection of the audio can be implemented.

[0080] In one embodiment, segmenting the audio data to obtain multiple audio sequences includes: performing frame segmentation on the audio data according to a preset frame length to obtain multiple audio sequences with the same frame length.

[0081] Specifically, the processor performs frame segmentation on the audio data according to the preset frame length to obtain audio sequences, and the frame lengths of all audio sequences are the same. By unifying the frame lengths of the audio sequences, subsequent data processing is facilitated. Among them, the number of frames offset between two adjacent audio sequences in the position of the audio data does not exceed the preset frame length to ensure that each frame of data in the audio data has corresponding data in the audio sequences.

[0082] In this embodiment, by segmenting the audio sequence, it is convenient to process the data in the subsequent audio sequence.

[0083] In one embodiment, detecting multiple audio sequences respectively includes: calculating the spectral energy of the audio sequence; detecting the multiple audio sequences according to the spectral energy.

[0084] Specifically, the processor calculates the spectral energy of the audio frequency points in the audio sequence, and determines whether there is abnormal audio data in the audio sequence by comparing with the preset audio spectral energy.

[0085] In this embodiment, by comparing the spectral energy, it is possible to effectively detect whether there is abnormal audio data in the audio sequence.

[0086] In one embodiment, if there is abnormal audio data in the audio sequence, obtaining an anomaly score for the audio sequence with abnormal audio data includes: extracting features from the audio sequence with abnormal audio data to obtain the first acoustic features of the audio sequence with abnormal audio data; invoking the first detection model, and inputting the first acoustic features into the first detection model to obtain the anomaly score.

[0087] Among them, the acoustic features may include at least one of time-domain features, frequency-domain features, and cepstral-domain features. The time-domain features may include feature vectors such as short-time energy, low-energy frame ratio, short-time zero-crossing rate, and high-zero-crossing rate frame ratio; the frequency-domain features may include feature vectors such as frequency-domain energy, center frequency, bandwidth, spectral roll-off coefficient, and linear prediction coefficient; the cepstral-domain features may include feature vectors such as Mel frequency cepstral coefficients and linear prediction cepstral coefficients. The time domain describes the relationship of a mathematical function or a physical signal with respect to time, the frequency domain describes the relationship of a mathematical function or a physical signal with respect to time, and the cepstral domain is a time domain obtained by performing a Fourier transform of a signal, followed by logarithmic operation and then inverse Fourier transform. The detection model refers to a model used to obtain the anomaly score.

[0088] The first detection model can be a memory-based anomaly detection model. This memory-based anomaly detection model can be as Figure 3As shown, by adding a preset number of abnormal audio sample data to a memory (i.e., the abnormal memory), the memory-based anomaly detection model can store the abnormal audio sample data, which plays an auxiliary role in the recognition process. This memory is independent of the autoencoder, and the memory-based anomaly detection model unifies the reconstruction error and similarity score of the autoencoder using an additive operation. The input sample is calculated for similarity with the abnormal audio sample data in the memory. If the input is a normal sample, the similarity will be relatively small; otherwise, the similarity value will be relatively large. Then, the similarity score is added to the anomaly degree score estimated by the anomaly detector as the anomaly score of the sound sample. Through the addition operation, even if the anomaly detector estimates a relatively low anomaly degree for the abnormal sound sample, that is, the anomaly score is small, but adding a large similarity score still makes the anomaly score of the abnormal sound sample large, thus distinguishing it from the normal sample. Among them, the scores of the two are combined in the form of a weighted sum. The weight used in the weighted sum is obtained by normalizing the anomaly scores of the machine samples used to train the memory-based anomaly detection model and then using the grid search method to obtain the optimal weight value.

[0089] Specifically, the processor normalizes the audio sequence with abnormal audio data, divides it into frames and applies a window function. Each frame of the signal calculates the spectrum of each frame through the discrete Fourier transform, and then combines the spectra of all frames to generate a two-dimensional time-frequency spectrogram. According to the two-dimensional time-frequency spectrogram, the first acoustic feature of the audio sequence with abnormal audio data is obtained. This first acoustic feature includes frequency domain features and Mel spectrum features, that is, the frequency domain features are obtained by logarithmically transforming the two-dimensional time-frequency spectrogram, and the Mel cepstral coefficients are obtained by passing the two-dimensional time-frequency spectrogram through a Mel filter and then performing a logarithmic transformation. Then, the Mel cepstral coefficients are further transformed through the discrete cosine transform to obtain the Mel cepstrum features; the first detection model is retrieved, and the frequency domain features, Mel spectrum features, and Mel cepstral coefficients are input into the first detection model to obtain the anomaly score of the audio sequence with abnormal audio data.

[0090] In this embodiment, by retrieving the first detection model, the anomaly score can be accurately calculated.

[0091] In one embodiment, retrieving the first detection model and inputting the first acoustic feature into the first detection model to obtain the anomaly score includes: calculating the first similarity between the first acoustic feature and the acoustic feature of the abnormal audio data in the memory; inputting the first acoustic feature into a preset anomaly detector to calculate the anomaly degree score; and obtaining the anomaly score using the first similarity and the anomaly degree score.

[0092] Specifically, the processor calculates a first similarity between the first acoustic feature and the acoustic feature of the abnormal audio data in the memory. The first acoustic feature is input into a preset abnormal detector, and the abnormal degree score is calculated according to the area enclosed by the Receiver Operating Characteristic (ROC) curve and the coordinate axes (Area Under Curve, AUC). The abnormal score is obtained by weighted sum using the first similarity and the abnormal degree score. Among them, the abscissa of the ROC curve is the false positive rate (also called the false positive class rate, False Positive Rate), which is related to the probability that normal audio is recognized as abnormal audio, and the ordinate of the ROC curve is the true negative rate (true negative class rate) or the detection rate, which is related to the probability that abnormal audio is accurately recognized.

[0093] In this embodiment, by obtaining the abnormal degree through the first similarity and the abnormal degree of abnormal detection, the accuracy of obtaining the abnormal score can be improved.

[0094] In one embodiment, the above audio detection method further includes: if there is no abnormal audio data in the audio sequence, acoustic feature extraction is performed on multiple audio sequences to obtain second acoustic features corresponding to each audio sequence in the multiple audio sequences; the second acoustic features are input into a second detection model, and a second similarity between the multiple audio sequences is calculated; the abnormal score is obtained according to the second similarity.

[0095] Among them, the second detection model can be a contrastive abnormal detection model. The contrastive abnormal detection model is trained based on abnormal sound samples, where the abnormal sound samples are obtained by changing the acoustic pattern of normal sound samples. Since the abnormal sound samples are closer to the normal sound samples and have slight differences, that is, for the second detection model, the abnormal sound samples are difficult samples, which are difficult to distinguish but can bring more information to the classifier. Geometrically speaking, the space represented by the classifier can just simulate the normal sample data distribution. Therefore, the data enhancement methods in this application are all based on normal sound samples, only changing the acoustic pattern of the normal samples, and regarding the newly generated samples as abnormal sound samples. The contrastive abnormal detection model can be as Figure 4As shown in the figure, an abnormal sound sample is generated by using the data augmentation method. The audio data input to the contrastive anomaly detection model is defined as a sample pair. The definition of the sample pair includes two categories. If both are normal audio data, it is defined as a positive sample pair, and its label is set to 0. If one is normal audio data and the other is abnormal audio data, it is defined as a negative sample pair, and its label is set to 1. The convolutional neural network can extract the discriminative information of the sample pair, and then calculate the similarity of the high-level semantic features of the input sample pair. Since the exact same network structure is used, similar samples are mapped to positions close to each other in the high-level space, while sample pairs with large differences are mapped to positions far apart. A contrast energy function can be used to reduce the contrast energy of similar sample pairs and increase the contrast energy of dissimilar sample pairs during the training process. The Euclidean distance is used to measure the difference between sample pairs in the high-dimensional embedding space. At the same time, instead of using the energy function as the loss function, it is processed through a fully connected layer after the distance vector difference operation, and the probability distribution is output through the sigmoid curve activation. This probability measures the similarity of the sample pair. If it is a positive sample pair, its similarity value is large, otherwise its similarity is small. Therefore, the label of the sample pair represents the similarity of the sample pair. Thus, the network can be trained through supervised binary classification learning, and the cross-entropy function is used as the loss function. The finally trained network can be used as a feature extractor to distinguish normal and abnormal sounds in the high-level space. Figure 4 The dotted line in the figure represents the inference stage. The network structure of the similarity part is removed, and only the feature extraction part is retained. During testing, the similarity of the high-level features of the sample to be tested and the normal sample is compared. If it is normal, the similarity is greater, otherwise it is smaller.

[0096] Specifically, if there is no abnormal audio data in the audio sequence, the acoustic features of multiple audio sequences are extracted to obtain the second acoustic features corresponding to each audio sequence in the multiple audio sequences. The second acoustic features are input into the second detection model. The Euclidean distance is used to measure the difference between the second acoustic features in the high-dimensional embedding space. After processing through a fully connected layer and then through the sigmoid curve activation, the probability distribution is output, and the second similarity between the multiple audio sequences is calculated. The negative value of the second similarity value is taken as the anomaly score.

[0097] In this embodiment, by obtaining the anomaly score through similarity, the accuracy of obtaining the anomaly score can be improved.

[0098] To better understand the audio detection method provided by the embodiments of the present application, the following describes the specific steps of the audio detection method in combination with a detailed embodiment, as Figure 5 shown, the specific steps of the audio detection method are described:

[0099] (1) The processor obtains the audio data to be detected;

[0100] (2) The processor performs frame segmentation on the audio data to be detected according to a preset frame length to obtain multiple audio sequences with the same frame length;

[0101] (3) The processor calculates the spectral energy of the audio frequency points of the audio sequence. If the spectral energy of the audio frequency points of the audio sequence is greater than the preset spectral energy, there is abnormal audio data in the audio sequence;

[0102] (4) If there is abnormal audio data in the audio sequence, the processor normalizes the audio sequence with abnormal audio data. After frame segmentation and windowing, the spectrum of each frame is calculated by discrete Fourier transform for each frame signal, and then the spectra of all frames are combined together to generate a two-dimensional time-frequency map. The two-dimensional time-frequency map is logarithmically transformed to obtain frequency domain features, and the two-dimensional time-frequency map passes through a Mel filter and is logarithmically transformed to obtain Mel cepstral coefficients. Then, the Mel cepstral coefficients are subjected to discrete cosine transform to obtain Mel cepstral features; the first detection model is retrieved, and the frequency domain features, Mel spectral features, and Mel cepstral coefficients are input into a preset anomaly detector. The anomaly degree score is calculated according to the area enclosed by the Receiver Operating Characteristic (ROC) curve and the coordinate axis (Area Under Curve, AUC); the processor calculates the first similarity between the first acoustic feature and the acoustic feature of the abnormal audio data in the memory, and uses the first similarity and the anomaly degree score to obtain the anomaly score through weighted sum, as Figure 3 shown;

[0103] (5) If there is no abnormal audio data in the audio sequence, the processor extracts acoustic features from the multiple audio sequences to obtain the second acoustic feature corresponding to each audio sequence in the multiple audio sequences; the second acoustic feature is input into the second detection model, and the convolutional neural network can extract the discriminative information of the second acoustic feature. The Euclidean distance is used to measure the difference of the second acoustic feature in the high-dimensional embedding space, and after processing by the fully connected layer and passing through the S-layer curve activation, the probability distribution is output, and the second similarity between the multiple audio sequences is calculated; the negative value of the second similarity value is taken as the anomaly score. As Figure 4 shown.

[0104] (6) The processor outputs a detection report according to the anomaly score.

[0105] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0106] Based on the same inventive concept, an embodiment of the present application further provides an audio detection device for implementing the above-mentioned audio detection method. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the audio detection device provided below can refer to the limitations on the audio detection method in the above text, and will not be repeated here.

[0107] In one embodiment, as Figure 7 shown, an audio detection device is provided, including: a data acquisition module 710, a processing module 720, a detection module 730, a score calculation module 740, and an output module 750, where:

[0108] The data acquisition module 710 is used to acquire audio data;

[0109] The processing module 720 is used to segment the audio data to obtain multiple audio sequences;

[0110] The detection module 730 is used to detect each of the multiple audio sequences separately;

[0111] The score calculation module 740 is used to calculate an abnormal score for the audio sequence with abnormal audio data if there is abnormal audio data in the audio sequence;

[0112] The output module 750 is used to output an audio detection report according to the abnormal score.

[0113] In the above audio detection device, audio data is acquired; the audio data is segmented to obtain multiple audio sequences; the multiple audio sequences are respectively detected; if there is abnormal audio data in an audio sequence, an abnormal score is obtained for the audio sequence with the abnormal audio data; and an audio detection report is output according to the abnormal score. By acquiring audio data, segmenting the audio data to obtain multiple audio sequences, detecting whether there is abnormal audio data in the audio sequences, obtaining the abnormal score for the audio sequence with the abnormal audio data, and then outputting the audio detection report, a comprehensive detection of the audio can be implemented.

[0114] In one embodiment, the processing module 720 is configured to perform frame segmentation on the audio data according to a preset frame length to obtain multiple audio sequences with the same frame length.

[0115] In one embodiment, the above audio detection device further includes: a calculation module. The calculation module is configured to calculate the spectral energy of the audio sequence; the detection module 730 is configured to detect the multiple audio sequences according to the spectral energy.

[0116] In one embodiment, the above audio detection device further includes: a feature extraction module. The feature extraction module is configured to extract features from the audio sequence with the abnormal audio data to obtain the first acoustic features of the audio sequence with the abnormal audio data; the score calculation module 740 is further configured to retrieve the first detection model and input the first acoustic features into the first detection model to obtain the abnormal score.

[0117] In one embodiment, the above audio detection device further includes: a calculation module. The calculation module is configured to calculate the first similarity between the first acoustic features and the acoustic features of the abnormal audio data in the memory; input the first acoustic features into a preset abnormal detector to calculate the abnormal degree score; the score calculation module 740 is further configured to use the first similarity and the abnormal degree score to obtain the abnormal score.

[0118] In one embodiment, the processing module 720 is configured to, if there is no abnormal audio data in the audio sequence, perform acoustic feature extraction on the multiple audio sequences to obtain the second acoustic features corresponding to each audio sequence in the multiple audio sequences; the calculation module is configured to input the second acoustic features into the second detection model to calculate the second similarity between the multiple audio sequences; the score calculation module 740 is configured to obtain the abnormal score according to the second similarity.

[0119] Each module in the above audio detection device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or be independent of it, or be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0120] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in Figure 8 . The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store noise data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an audio detection method.

[0121] Those skilled in the art can understand that Figure 8 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0123] Obtain audio data;

[0124] Perform segmentation processing on the audio data to obtain multiple audio sequences;

[0125] Detect each of the multiple audio sequences separately;

[0126] If there is abnormal audio data in the audio sequence, calculate an abnormal score for the audio sequence with abnormal audio data;

[0127] Output an audio detection report according to the abnormal score.

[0128] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0129] Perform framing processing on the audio data according to a preset frame length to obtain multiple audio sequences with the same frame length.

[0130] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0131] Calculate the spectral energy of the audio sequence;

[0132] Detect the multiple audio sequences according to the spectral energy.

[0133] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0134] Extract features from the audio sequence with abnormal audio data to obtain the first acoustic features of the audio sequence with abnormal audio data;

[0135] Retrieve the first detection model, and input the first acoustic features into the first detection model to obtain an anomaly score.

[0136] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0137] Calculate the first similarity between the first acoustic features and the acoustic features of the abnormal audio data in the memory;

[0138] Input the first acoustic features into a preset anomaly detector to calculate the anomaly degree score;

[0139] Use the first similarity and the anomaly degree score to obtain the anomaly score.

[0140] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0141] If there is no abnormal audio data in the audio sequence, extract the acoustic features of multiple audio sequences to obtain the second acoustic features corresponding to each audio sequence in the multiple audio sequences;

[0142] Input the second acoustic features into the second detection model to calculate the second similarity between the multiple audio sequences;

[0143] Obtain the anomaly score according to the second similarity.

[0144] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0145] Obtain audio data;

[0146] Segment the audio data to obtain multiple audio sequences;

[0147] Detect multiple audio sequences respectively;

[0148] If there is abnormal audio data in the audio sequence, obtain the anomaly score for the audio sequence with abnormal audio data;

[0149] Output an audio detection report according to the anomaly score.

[0150] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0151] Frame the audio data according to a preset frame length to obtain multiple audio sequences of the same frame length.

[0152] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0153] Calculate the spectral energy of the audio sequence;

[0154] Detect the multiple audio sequences according to the spectral energy.

[0155] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0156] Frame the audio data according to a preset frame length to obtain multiple audio sequences of the same frame length.

[0157] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0158] Calculate the spectral energy of the audio sequence;

[0159] Detect the multiple audio sequences according to the spectral energy.

[0160] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0161] Extract features from the audio sequence with abnormal audio data to obtain the first acoustic features of the audio sequence with abnormal audio data;

[0162] Retrieve the first detection model, and input the first acoustic features into the first detection model to obtain an anomaly score.

[0163] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0164] Calculate the first similarity between the first acoustic features and the acoustic features of the abnormal audio data in the memory;

[0165] Input the first acoustic features into a preset anomaly detector to calculate the anomaly degree score;

[0166] Use the first similarity and the anomaly degree score to obtain the anomaly score.

[0167] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0168] If there is no abnormal audio data in the audio sequence, extract the acoustic features of the multiple audio sequences to obtain the second acoustic features corresponding to each audio sequence in the multiple audio sequences;

[0169] Input the second acoustic feature into the second detection model, and calculate the second similarity between multiple audio sequences;

[0170] Obtain the anomaly score according to the second similarity.

[0171] In one embodiment, a computer program product is provided, including a computer program which, when executed by a processor, implements the following steps:

[0172] Obtain audio data;

[0173] Segment the audio data to obtain multiple audio sequences;

[0174] Detect each of the multiple audio sequences separately;

[0175] If there is abnormal audio data in the audio sequence, obtain the anomaly score for the audio sequence with abnormal audio data;

[0176] Output an audio detection report according to the anomaly score.

[0177] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0178] According to a preset frame length, perform frame segmentation on the audio data to obtain multiple audio sequences with the same frame length.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0180] Calculate the spectral energy of the audio sequence;

[0181] Detect the multiple audio sequences according to the spectral energy.

[0182] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0183] Extract features from the audio sequence with abnormal audio data to obtain the first acoustic feature of the audio sequence with abnormal audio data;

[0184] Retrieve the first detection model, and input the first acoustic feature into the first detection model to obtain the anomaly score.

[0185] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0186] Calculate the first similarity between the first acoustic feature and the acoustic feature of the abnormal audio data in the memory;

[0187] Input the first acoustic feature into a preset anomaly detector to calculate the anomaly degree score;

[0188] Calculate the anomaly score using the first similarity and the anomaly degree score.

[0189] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0190] If there is no abnormal audio data in the audio sequence, perform acoustic feature extraction on the multiple audio sequences to obtain the second acoustic features corresponding to each audio sequence in the multiple audio sequences;

[0191] Input the second acoustic features into the second detection model to calculate the second similarity between the multiple audio sequences;

[0192] Calculate the anomaly score according to the second similarity.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0194] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0195] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0196] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An audio detection method, characterized in that, The method includes: Obtain audio data; Perform segmentation processing on the audio data to obtain a plurality of audio sequences; Detect each of the plurality of audio sequences separately; If there is abnormal audio data in the audio sequence, calculate an abnormal score for the audio sequence with the abnormal audio data. The abnormal audio data refers to audio information in the audio that is easily confused and causes a negative impact, and the abnormal score represents the degree of abnormality of the audio sequence with the abnormal audio data; Output an audio detection report according to the abnormal score; The step of, if there is abnormal audio data in the audio sequence, calculating an abnormal score for the audio sequence with the abnormal audio data, includes: Extract features from the audio sequence with the abnormal audio data to obtain the first acoustic features of the audio sequence with the abnormal audio data; Retrieve a first detection model, and input the first acoustic features into the first detection model to calculate the abnormal score. The first detection model is a memory-based anomaly detection model.

2. The method according to claim 1, characterized in that, The step of performing segmentation processing on the audio data to obtain a plurality of audio sequences includes: Perform frame segmentation on the audio data according to a preset frame length to obtain a plurality of audio sequences with the same frame length.

3. The method according to claim 1, characterized in that, The step of detecting each of the plurality of audio sequences separately includes: Calculate the spectral energy of the audio sequence; Detect the plurality of audio sequences through the spectral energy.

4. The method according to claim 1, wherein The first acoustic features include frequency domain features and mel cepstral features. The step of extracting features from the audio sequence with the abnormal audio data to obtain the first acoustic features of the audio sequence with the abnormal audio data includes: Generate a two-dimensional time-frequency spectrogram according to the audio sequence with the abnormal audio data; Obtain frequency domain features by performing logarithmic transformation on the two-dimensional time-frequency spectrogram; Pass the two-dimensional time-frequency spectrogram through a mel filter and perform logarithmic transformation to obtain mel cepstral coefficients; Obtain mel cepstral features by performing discrete cosine transform on the mel cepstral coefficients; The step of retrieving a first detection model and inputting the first acoustic features into the first detection model to calculate the abnormal score includes: Retrieve a first detection model, and input the first acoustic features and the mel cepstral coefficients into the first detection model to calculate the abnormal score.

5. The method according to claim 1, characterized in that, The step of retrieving a first detection model and inputting the first acoustic features into the first detection model to calculate the abnormal score includes: Calculate a first similarity between the first acoustic features and the acoustic features of the abnormal audio data in the memory; Input the first acoustic features into a preset anomaly detector to calculate an anomaly degree score; Calculate the abnormal score by using the first similarity and the anomaly degree score.

6. The method according to claim 1, wherein The above method further includes: If there is no abnormal audio data in the audio sequence, extract acoustic features from the plurality of audio sequences to obtain second acoustic features corresponding to each audio sequence in the plurality of audio sequences; Input the second acoustic features into a second detection model, and calculate a second similarity between the plurality of audio sequences; Calculate the abnormal score according to the second similarity.

7. An audio detection device, characterized in that, The device includes: A data acquisition module for obtaining audio data; A processing module, configured to segment the audio data to obtain a plurality of audio sequences; A detection module, configured to respectively detect the plurality of audio sequences; A score calculation module, configured to calculate an anomaly score for the audio sequence with anomaly audio data if there is anomaly audio data in the audio sequence, where the anomaly audio data refers to audio information that is easily confused and has a negative impact in the audio, and the anomaly score represents the anomaly degree of the audio sequence with the anomaly audio data; An output module, configured to output an audio detection report according to the anomaly score; The step of calculating an anomaly score for the audio sequence with anomaly audio data if there is anomaly audio data in the audio sequence includes: Performing feature extraction on the audio sequence with anomaly audio data to obtain a first acoustic feature of the audio sequence with anomaly audio data; Invoking a first detection model, and inputting the first acoustic feature into the first detection model to calculate the anomaly score, where the first detection model is a memory-based anomaly detection model.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images