Abnormal sound classification method, system, device and storage medium

Through the multi-level classification method, the multi-dimensional characteristics of the audio signal are used for preliminary detection and further classification of binary classifiers, which solves the problem that the inability to accurately identify sound types in the prior art, and improves the accuracy and robustness of abnormal sound detection sex.

CN113689888BActive Publication Date: 2024-12-06ZHEJIANG DAHUA TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110871836.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-30
Publication Date
2024-12-06
Estimated Expiration
2041-07-30

AI Technical Summary

Technical Problem

The existing abnormal audio detection methods cannot accurately identify the types of sounds that are prone to confusion in actual conference scenarios, such as explosions, gunshots, screams, etc., resulting in low accuracy of abnormal sound detection.

Method used

Using a multi-level classification method, by obtaining the time domain, frequency domain and cepspectral domain characteristics of the audio signal, input it into a multi-classifier for preliminary detection, establish a binary classifier based on the maximum likelihood difference value, perform further classification, and reduce Small similar sound recognition interference.

Benefits of technology

It significantly improves the accuracy of abnormal sound detection, reduces the interference of similar sounds on recognition, and enhances the robustness of the system and the multi-angle analysis capability of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113689888B_ABST
    Figure CN113689888B_ABST
Patent Text Reader

Abstract

The present application discloses an abnormal sound classification method, system, device and storage medium, the method comprising: obtaining the acoustic features of the abnormal audio signal to be detected; inputting the acoustic features into a multi-classifier, using the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features; in response to the difference between the two largest likelihoods being less than a set value, using the two-classifier of the two abnormal categories corresponding to the two largest likelihoods to detect the acoustic features, and obtain the abnormal type of the abnormal audio signal. In the above manner, the present application can greatly reduce the interference of similar sounds on the recognition of abnormal audio signals, thereby improving the accuracy of abnormal sound detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of sound signal processing, and in particular to an abnormal sound classification method, system, device and storage medium. Background Art

[0002] In the security field, the mainstream monitoring method is video monitoring. However, due to its limited monitoring field, video monitoring cannot detect events in blind spots. Therefore, the use of audio event detection methods as an auxiliary in the security field will be more efficient in detecting events that threaten public safety. The combination of video monitoring and audio monitoring is the future development trend.

[0003] After obtaining the audio clip to be detected, the current abnormal audio detection method divides the audio clip into frames to extract features in the time domain, frequency domain or cepstrum domain, and then sends the feature clip to the classification model, which identifies the feature clip to obtain the final detection result.

[0004] The existing technology only performs a simple classification of feature fragments. If the audio fragment to be detected belongs to one of multiple sound types that are easily confused, for example, in an actual meeting scenario, it is necessary to detect explosions, gunshots, screams and other public safety threats. However, since keyboard typing and door closing sounds are easily identified as gunshots, footsteps are also easily confused with gunshots, and coughs and screams have certain similarities, the existing technology cannot accurately determine the abnormal category to which the audio fragment belongs, and cannot improve the accuracy of abnormal sound detection. Summary of the invention

[0005] The main technical problem solved by the present application is to provide a method, system, device and storage medium for classifying abnormal sound, which can improve the accuracy of abnormal sound detection by performing multi-level classification of abnormal audio signals.

[0006] To solve the above technical problems, the first technical solution adopted in the present application is to provide an abnormal sound classification method, including: obtaining the acoustic features of the abnormal audio signal to be detected; inputting the acoustic features into a multi-classifier, and using the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features; in response to the difference between the two largest likelihoods being less than a set value, using a binary classifier of two abnormal categories corresponding to the two largest likelihoods to detect the acoustic features, and obtain the abnormal type of the abnormal audio signal.

[0007] Among them, the step of obtaining the acoustic features of the abnormal audio signal to be detected includes: obtaining the abnormal audio signal to be detected; performing multi-dimensional feature extraction on the abnormal audio signal to obtain the multi-dimensional acoustic features of the abnormal audio signal; wherein the multi-dimensional acoustic features include at least two of time domain features, frequency domain features and cepstrum domain features.

[0008] Among them, the step of obtaining the acoustic features of the abnormal audio signal to be detected includes: obtaining the abnormal audio signal and performing frame processing on the abnormal audio signal; extracting features from each frame of the abnormal audio signal to obtain the acoustic features of each frame of the abnormal audio signal; inputting the acoustic features into a multi-classifier, and using the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features, including: inputting the acoustic features of each frame of the abnormal audio signal into the multi-classifier to obtain the likelihood of each frame of the abnormal audio signal and various abnormal categories; in response to the difference between the two largest likelihoods being less than a set value, using the binary classifiers of the two abnormal categories corresponding to the two largest likelihoods to detect the acoustic features to obtain the abnormal type of the abnormal audio signal, including: in response to the difference between the two largest likelihoods of the current frame being less than a set value, using the binary classifiers of the two abnormal categories corresponding to the two largest likelihoods to detect the current frame to obtain the abnormal type of the current frame; using the abnormal types of a set number of consecutive frames of abnormal audio signals to determine the abnormal type of the abnormal audio signal.

[0009] Among them, the step of determining the abnormal type of the abnormal audio signal using the abnormal types of a plurality of frames of abnormal audio signals of a continuous set amount includes: counting the number of frames of abnormal audio signals belonging to each abnormal type; and determining the abnormal type including the largest number of frames as the abnormal type of the abnormal audio signal.

[0010] Among them, the step of extracting features from each frame of abnormal audio signal to obtain the acoustic features of each frame of abnormal audio signal includes: performing windowing processing on each frame of abnormal audio signal; transforming the time domain signal after windowing processing into a frequency domain signal by using fast Fourier transform, and then obtaining a signal energy spectrum based on the frequency domain signal; performing bandpass filtering on the signal energy spectrum by using a filter, and taking the logarithm of the output of the filter to transform it into a logarithmic energy spectrum domain; transforming the logarithmic energy spectrum domain into a cepstrum domain by using discrete cosine transform, and obtaining feature vectors that are independent of each other in each dimension.

[0011] Among them, in response to the difference between the two maximum likelihoods being less than a set value, the acoustic features are detected using a binary classifier of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the abnormal audio signal, including: in response to the difference between the two maximum likelihoods being less than a set value, two abnormal categories corresponding to the two maximum likelihoods are extracted to establish a binary classifier; wherein the binary classifier includes a Gaussian kernel function; the acoustic features of the abnormal audio signal are input into the binary classifier, and statistical features with a longer distance are extracted based on clustering using the Gaussian kernel function, and the acoustic features are classified based on the statistical features to obtain the abnormal type of the abnormal audio signal.

[0012] Among them, before the step of obtaining the acoustic characteristics of the abnormal audio signal to be detected, it also includes: obtaining the audio signal, calculating the short-time energy of the audio signal; comparing the short-time energy with the set energy threshold; in response to the short-time energy of the audio signal being greater than the set energy threshold, determining the audio signal as the abnormal audio signal to be detected.

[0013] To solve the above technical problems, the second technical solution adopted in the present application is to provide an abnormal sound classification system, including: a feature acquisition module, used to acquire the acoustic features of the abnormal audio signal to be detected; a first classification module, used to input the acoustic features into a multi-classifier, and use the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features; a second classification module, used to respond to the difference between the two largest likelihoods being less than a set value, and use the binary classifiers of the two abnormal categories corresponding to the two largest likelihoods to detect the acoustic features to obtain the abnormal type of the abnormal audio signal.

[0014] To solve the above technical problems, the third technical solution adopted in the present application is to provide an abnormal sound classification device, comprising: a memory for storing program data, which implements the steps in the abnormal sound classification method as described in any one of the above when the stored program data is executed; and a processor for executing program instructions stored in the memory to implement the steps in the abnormal sound classification method as described in any one of the above.

[0015] In order to solve the above technical problems, the fourth technical solution adopted in the present application is to provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the abnormal sound classification method as described in any one of the above items are implemented.

[0016] The beneficial effect of the present application is as follows: Different from the prior art, the present application provides an abnormal sound classification method, system, device and storage medium. By inputting the acquired acoustic features of the abnormal audio signal to be detected into a multi-classifier for preliminary detection, the approximate abnormal type of the abnormal audio signal can be predicted, and then a more targeted binary classifier is established based on the results of the preliminary detection, and the acoustic features are input into the binary classifier for classification, which can greatly reduce the interference of similar sounds on the recognition of abnormal audio signals, thereby improving the accuracy of abnormal sound detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 It is a flowchart of an implementation method of an abnormal sound classification method of the present application;

[0019] Figure 2 It is a flowchart of an implementation method of the present application for obtaining an abnormal audio signal to be detected;

[0020] Figure 3 It is a flowchart of a specific implementation method of extracting acoustic features in this application;

[0021] Figure 4 yes Figure 3 A flowchart of a specific implementation method of step S32;

[0022] Figure 5 yes Figure 1 A flowchart of a specific implementation method of step S13;

[0023] Figure 6 It is a structural schematic diagram of an implementation scheme of an abnormal sound classification system of the present application;

[0024] Figure 7 It is a structural schematic diagram of an embodiment of an abnormal sound classification device of the present application;

[0025] Figure 8 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0027] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said", and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless otherwise clearly indicated above, and "multiple" generally includes at least two, but does not exclude the inclusion of at least one.

[0028] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0029] It should be understood that the terms "include", "comprises" or any other variations used herein are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "includes..." do not exclude the presence of other identical elements in the process, method, article or device that includes the elements.

[0030] Existing abnormal audio detection methods only perform simple classification of the extracted sound feature fragments. If the audio fragment to be detected belongs to one of multiple sound types that are easily confused, for example, in actual meeting scenarios, it is necessary to detect explosions, gunshots, screams and other public safety threats. However, since keyboard tapping and door closing sounds are easily identified as gunshots, footsteps are also easily confused with gunshots, and coughs and screams have certain similarities, the existing technology cannot accurately determine the abnormal category to which the audio fragment belongs, and cannot improve the accuracy of abnormal sound detection.

[0031] Based on the above situation, the present application provides an abnormal sound classification method, system, device and storage medium, which can improve the accuracy of abnormal sound detection by performing multi-level classification of abnormal audio signals.

[0032] The present application is described in detail below with reference to the accompanying drawings and implementation methods.

[0033] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an embodiment of an abnormal sound classification method of the present application. Figure 1 As shown, in this embodiment, the method includes:

[0034] S11: Acquire the acoustic features of the abnormal audio signal to be detected.

[0035] In this implementation, the abnormal audio signal to be detected is firstly acquired.

[0036] Specifically, see Figure 2 , Figure 2 FIG. 1 is a flow chart of an implementation method of the present invention for obtaining an abnormal audio signal to be detected. Figure 2 As shown, in this embodiment, the method includes:

[0037] S21: Acquire an audio signal and calculate the short-time energy of the audio signal.

[0038] In this implementation, the audio signal can be obtained from audio monitoring.

[0039] Short-time energy is a type of time domain feature, which reflects the energy of the audio signal. If the calculated short-time energy of the audio signal is high, it indicates that the energy of the audio signal is high.

[0040] In this embodiment, a large-scale rough inspection is performed on the audio signal by endpoint detection to detect the starting point candidate area and the ending point candidate area of ​​the audio signal, and then the candidate area is finely inspected to detect the endpoints of the audio signal, and the short-time energy calculation is performed on the endpoints.

[0041] In a surveillance environment, most of the surveillance scenes are relatively quiet, the audio stream is stable and has a low energy value. If an incident threatening public safety occurs, the energy of the endpoints in the selected area is strong. By calculating the short-time energy of the endpoints, it is possible to detect whether an abnormal event occurs in the audio signal.

[0042] In other implementations, detection can also be performed by calculating the short-time zero-crossing rate of the audio signal. The short-time zero-crossing rate is also a type of time domain feature. The short-time zero-crossing rate reflects the jitter degree of the audio signal. If the calculated short-time zero-crossing rate is high, it indicates that the audio signal jitter is more obvious.

[0043] The above method determines sound abnormality through short-time energy and short-time zero-crossing rate. Since short-time energy and short-time zero-crossing rate are both time domain features, they do not involve frequency domain changes and calculation of characteristic parameters, thus reducing the complexity of calculation.

[0044] S22: Compare the short-time energy with the set energy threshold.

[0045] In this implementation manner, the set energy threshold may be a threshold set by the user based on experience, or may be a value obtained through self-learning of sound.

[0046] Among them, if the short-term energy of the endpoint is greater than the set energy threshold, it indicates that the probability of an abnormal event is high, otherwise, it is normal.

[0047] S23: In response to the short-term energy of the audio signal being greater than a set energy threshold, determining the audio signal as an abnormal audio signal to be detected.

[0048] In a specific implementation scenario, in response to the short-time energy of the audio signal being greater than the set energy threshold, it indicates that the audio signal in the area is abnormal, and the probability of an abnormal event is high, and it is determined as an abnormal audio signal to be detected, and the next step of detection is performed to identify the abnormal category. Among them, in response to the short-time energy of the audio signal being less than the set energy threshold, it indicates that the audio signal in the area is normal and does not need to be detected in the next step.

[0049] In this implementation, after the abnormal audio signal to be detected is acquired, multi-dimensional feature extraction is performed on the abnormal audio signal to obtain multi-dimensional acoustic features of the abnormal audio signal.

[0050] The multi-dimensional acoustic features include at least two of time domain features, frequency domain features and cepstrum domain features.

[0051] Among them, the time domain features include feature vectors such as short-time energy, low-energy frame ratio, short-time zero-crossing rate, and high zero-crossing rate frame ratio; the frequency domain features include feature vectors such as frequency domain energy, frequency center, bandwidth, spectrum roll-off coefficient, and linear prediction coefficient; the cepstrum domain features include feature vectors such as Mel-frequency cepstrum coefficients and linear prediction cepstrum coefficients.

[0052] Specifically, the time domain describes the relationship between mathematical functions or physical signals and time, the frequency domain describes the relationship between mathematical functions or physical signals and time, and the cepstrum domain is the time domain obtained by performing a Fourier transform of a signal after logarithmic operation and then inverse Fourier transform.

[0053] Since the sound signal is originally a one-dimensional time domain signal, it is difficult to intuitively see the frequency change pattern. The sound signal can be transformed into the frequency domain through Fourier transform. Although the frequency distribution of the signal can be seen through the frequency, the change of the frequency distribution over time cannot be seen due to the loss of time domain information. The cepstrum domain is the time domain obtained by taking the logarithm of the raw spectrogram and then transforming it back through discrete Fourier transform. It is not a time domain in the full sense, but has the characteristics of both the time domain and the frequency domain. It can see both the time domain information and the change of frequency over time. The cepstrum domain characteristics can be used to obtain the process from the occurrence to the change of abnormal sound events.

[0054] Specifically, see Figure 3 , Figure 3 FIG. 1 is a flow chart of a specific implementation method of extracting acoustic features in this application. Figure 3 As shown, in this embodiment, the method includes:

[0055] S31: Acquire an abnormal audio signal and perform frame processing on the abnormal audio signal.

[0056] The process of obtaining the abnormal audio signal can be referred to steps S21 to S23, which will not be described in detail here.

[0057] Overall, the abnormal audio signal is a long-term non-stationary signal, and its characteristics change over time. However, within a short time range, for example, within a short time of 10 to 50 ms, its characteristics remain basically unchanged, relatively stable, and have short-term stability. Therefore, this implementation method intercepts the long-term non-stationary signal into a short-term stable signal for processing. In a specific implementation scenario, the abnormal audio signal can be framed using a 40 ms frame length and a 20 ms frame shift.

[0058] Due to the advantage of the Hamming window function for boundary points in the process of intercepting signals, this implementation adopts the Hamming window as the window function to frame the abnormal audio signal.

[0059] Specifically, the calculation formula of the Hamming window function is as follows:

[0060]

[0061] Among them, w(n) is the window function, cos is the cosine function, and π is pi.

[0062] S32: Extract features of each frame of abnormal audio signal to obtain acoustic features of each frame of abnormal audio signal.

[0063] Specifically, see Figure 4 , Figure 4 yes Figure 3 Schematic diagram of a specific implementation of step S32 in FIG. Figure 4 As shown, in this embodiment, the step of extracting features from each frame of abnormal audio signal to obtain acoustic features of each frame of abnormal audio signal specifically includes:

[0064] S41: Perform windowing processing on each frame of abnormal audio signal.

[0065] In this implementation, a Hamming window function is used to perform windowing processing on each frame of abnormal audio signal.

[0066] In a specific implementation scenario, assuming that the abnormal audio signal is s(n), the frame signal after the window function is s(n)*w(n).

[0067] S42: transform the windowed time domain signal into a frequency domain signal using a fast Fourier transform, and then obtain a signal energy spectrum based on the frequency domain signal.

[0068] In this implementation, the time domain signal of the frame signal is transformed into a frequency domain signal using Fast Fourier Transform (FFT), and then the square of the spectrum amplitude is calculated to obtain the signal energy spectrum.

[0069] In this implementation, time domain features and frequency domain features can be acquired through time domain signals and frequency domain signals.

[0070] S43: using a filter to perform bandpass filtering on the signal energy spectrum, and taking the logarithm of the output of the filter to transform it into a logarithmic energy spectrum domain.

[0071] In this embodiment, a group of triangular filters are used to perform bandpass filtering on the energy spectrum at frequency. The function of this group of filters is to map the frequency domain space of the sound signal to the frequency domain space of human perception through the Mel scale, so that the spatial scale of the filter is close to the perception scale of human hearing.

[0072] Furthermore, the output of the filter is logarithmically transformed into the logarithmic energy spectrum domain. The advantage of this is that the convolution component of the speech signal in the time domain and the multiplicative component in the linear energy spectrum domain can be transformed into the additive component in the logarithmic energy spectrum domain, which is conducive to eliminating the adverse effects of the environment and channel. Compared with the linear energy spectrum domain, the statistical distribution characteristics of the logarithmic energy spectrum domain are more suitable for statistical modeling.

[0073] S44: Using discrete cosine transform, the logarithmic energy spectrum domain is transformed into the cepstrum domain to obtain independent feature vectors in each dimension.

[0074] In this implementation, discrete cosine transform (DCT) is used to transform the logarithmic energy spectrum domain into the cepstrum domain. The main purpose of this is to decorrelate the spectral components of different frequency bands to obtain independent feature vectors of each dimension, which is convenient for further modeling and calculation, and the cepstrum parameters are more compact and concise in form.

[0075] Furthermore, in order to reflect the occurrence and change process of abnormal sound events, that is, the dynamic characteristics of speech, this embodiment also adds first-order and second-order difference cepstrum to the acoustic features to obtain MFCC (Mel Cepstral Coefficient) features.

[0076] Among them, MFCC features are commonly used features in speech recognizers and are acoustic features extracted based on the human auditory system.

[0077] It can be understood that multi-dimensional features in the time domain, frequency domain and cepstrum domain can be extracted by the above method. This embodiment can analyze abnormal audio signals from multiple angles by combining the features of multiple domains for detection, thereby significantly improving the detection and recognition rate of the classification system. Furthermore, since the detected acoustic features are multi-dimensional, the robustness of the classification system can also be enhanced.

[0078] S12: Input the acoustic features into a multi-classifier, and use the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features.

[0079] In this embodiment, the multi-classifier is a trained GMM (Gaussian mixture model) classifier composed of 20-dimensional MFCC and its first-order and second-order differential features, linear prediction cepstral coefficients, time domain features, frequency domain features and spectral entropy, a total of 63-dimensional features. Among them, the GMM multi-classifier uses MFCC as a classification feature and establishes multiple Gaussian mixture models for various abnormal sound events.

[0080] The Gaussian mixture model is a semi-parametric density estimation method, which is obtained by weighted summation of several Gaussian probability density functions. Each Gaussian model is called a Gaussian component. The Gaussian mixture function model is:

[0081]

[0082] Among them, p(x|λ) represents the Gaussian mixture model, x represents the feature vector, λ represents the parameter set of the feature vector, and i represents the hidden state number. Represents the weight value of the i-th Gaussian component, satisfying the constraints p(x|i,λ) represents the probability density function of the hidden state.

[0083] The specific expression of p(x|i,λ) is as follows:

[0084]

[0085] Among them, d represents the dimension, π represents the circumference, μ represents the i represents the mean vector of the i-th Gaussian component, ∑ i represents the covariance matrix corresponding to the i-th Gaussian component, and T represents the transpose.

[0086] It can be seen that the parameter set of the Gaussian mixture model is The calculation of the Gaussian mixture model parameter set is mainly estimated through the expectation maximization algorithm, that is, when calculating a certain parameter, the other two parameters are fixed as constants, and then the derivative of the required variable is obtained.

[0087] Specifically, each feature vector is modeled as a single Gaussian, and an abnormal sound event usually includes multiple feature vectors. In the training process of the GMM multi-classifier, a splitting method is used to establish a Gaussian mixture model corresponding to the feature components included in each type of abnormal sound event. Since the feature vectors (model parameter quantities) required to distinguish different abnormal sound events are different, the number of Gaussian components of each abnormal sound event finally established is different.

[0088] In this implementation, the acoustic features of each frame of abnormal audio signal are input into a multi-classifier to obtain the likelihood of each frame of abnormal audio signal and various abnormal categories.

[0089] Specifically, the multi-dimensional acoustic features of each frame of abnormal audio signal are input into the GMM multi-classifier, and all feature vectors included in the acoustic features of the current frame are used as x input to obtain multiple posterior probabilities relative to various Gaussian mixture models. The abnormal sound event corresponding to the Gaussian mixture model with the largest posterior probability is the most likely abnormal type predicted to correspond to the current frame.

[0090] S13: In response to the difference between the two maximum likelihoods being less than a set value, the acoustic features are detected using binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain an abnormal type of the abnormal audio signal.

[0091] In this implementation, in response to the difference between the two maximum likelihoods of the current frame being less than a set value, the current frame is detected using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the current frame.

[0092] Specifically, the GMM multi-classifier predicts the likelihood of the current frame and each Gaussian mixture model and sorts the calculated likelihoods.

[0093] Among them, responding to the two largest likelihoods refers to the top two likelihoods. In response to the difference between the two largest likelihoods being less than a set value, it indicates that the abnormal sound events corresponding to the two top two Gaussian mixture models are both likely to be the abnormal type actually corresponding to the abnormal audio signal of the current frame. In response to the difference between the two largest likelihoods being greater than a set value, it indicates that the abnormal sound event corresponding to the Gaussian mixture model ranked first is very likely to be the abnormal type actually corresponding to the abnormal audio signal of the current frame.

[0094] The GMM multi-classifier matches abnormal audio signals based on multiple Gaussian mixture models, and can perform preliminary inspections on abnormal audio signals to predict the approximate abnormal type of abnormal audio signals. Two categories with very close likelihoods calculated by the GMM multi-classifier for an input feature can no longer be accurately determined simply by changing the GMM multi-classifier.

[0095] For example, in an actual meeting scenario, it is necessary to detect events that threaten public safety, such as explosions, gunshots, screams, and glass shattering. Since keyboard typing, door closing, and glass breaking are all easily identified as gunshots, and footsteps are also easily confused with gunshots, keyboard typing, glass shattering, door opening and closing, talking, and coughing are all likely to be identified as gunshots after being input into the GMM multi-classifier, thereby reducing the accuracy of abnormal sound detection.

[0096] In this implementation, for two types of abnormal sound events that are easily confused, a trained binary classifier is used to distinguish the abnormal audio signal of the current frame.

[0097] Specifically, see Figure 5 , Figure 5 yes Figure 1 Schematic diagram of a specific implementation of step S13 in FIG. Figure 5 As shown, in this embodiment, in response to the difference between the two maximum likelihoods being less than a set value, the acoustic features are detected using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the abnormal audio signal, specifically including:

[0098] S51: In response to the difference between the two maximum likelihoods being less than a set value, extracting two abnormal categories corresponding to the two maximum likelihoods to establish a binary classifier; wherein the binary classifier includes a Gaussian kernel function.

[0099] In this embodiment, in response to the difference between the two maximum likelihoods of the current frame being less than a set value, two abnormal categories corresponding to the two maximum likelihoods are extracted to establish an SVM (Support Vector Machines) binary classifier, and the features that are most likely to distinguish the two abnormal sound events are extracted to train the SVM binary classifier.

[0100] Among them, the SVM binary classifier includes a Gaussian kernel function.

[0101] Specifically, the support vector machine is a binary classification model, which can achieve nonlinear classification for linearly inseparable training data by using the Gaussian kernel function. The features used to train the SVM binary classifier are generated by clustering, and clustering can be used to find statistical features with a long distance between two types of abnormal sound events, that is, statistical features that are easy to distinguish between the two types of abnormal sound events.

[0102] In order to reduce the complexity of the model while ensuring the prediction accuracy of the SVM binary classifier, some features that are easier to count can be used as training features. For example, when the two largest likelihoods correspond to the two abnormal categories of keyboard tapping and gunshots, the feature set that can be used is: short-time zero-crossing rate, frequency center, bandwidth, spectral entropy, short-time energy, spectral roll-off coefficient, and low-energy frame ratio. When the two largest likelihoods correspond to the two abnormal categories of walking and gunshots, the feature set that can be used is: bandwidth, short-time zero-crossing rate, short-time energy, frequency domain energy, spectral roll-off coefficient, frequency domain peak factor, and short-time autocorrelation. When the two largest likelihoods correspond to the two abnormal categories of coughing and screaming, the feature set that can be used is: Mel frequency cepstrum coefficients, high zero-crossing rate frame ratio, short-time energy, and other acoustic features.

[0103] In this implementation, the trained SVM binary classifier can better accurately judge two abnormal categories with very close likelihoods, so as to reduce the interference of similar sounds on the recognition of abnormal audio signals.

[0104] S52: Input the acoustic features of the abnormal audio signal into a binary classifier, extract statistical features with a longer distance based on clustering using a Gaussian kernel function, and classify the acoustic features based on the statistical features to obtain the abnormal type of the abnormal audio signal.

[0105] In this embodiment, the acoustic features of the abnormal audio signal of the current frame are input into a binary classifier, and the statistical features that are most likely to distinguish the two abnormal categories are extracted based on clustering using a Gaussian kernel function. The extracted statistical features are classified using an SVM binary classifier to obtain the abnormal type of the abnormal audio signal of the current frame.

[0106] In order to reduce the false alarm rate of the SVM binary classifier and further improve the prediction accuracy of the SVM binary classifier, this embodiment uses the abnormal types of a set number of consecutive frames of abnormal audio signals to determine the abnormal type of the abnormal audio signal. Specifically, this embodiment counts the number of frames of abnormal audio signals belonging to each abnormal type, and determines the abnormal type with the largest number of frames as the abnormal type of the abnormal audio signal.

[0107] The set amount may be 3 frames, 5 frames or more frames, which is not limited in this application.

[0108] In a specific implementation scenario, three frames of abnormal audio signals continuously input are classified through an SVM binary classifier. If in the recognition results of the previous and next two frames and the current frame, the abnormal types of the abnormal audio signals of the previous and next two frames are both identified as type A, and the abnormal type of the abnormal audio signal of the current frame is identified as type B, then the abnormal type of the abnormal audio signal of the current frame is smoothed to type A through voting.

[0109] In this implementation, in order to improve the prediction accuracy of the GMM multi-classifier, in response to the difference between the two largest likelihoods being greater than a set value, the abnormal type of the abnormal audio signal is also determined by using the abnormal types of a set number of consecutive frames of abnormal audio signals.

[0110] In another specific implementation scenario, three frames of abnormal audio signals continuously input are classified through the GMM multi-classifier. If in the recognition results of the previous and next two frames and the current frame, the abnormal types of the abnormal audio signals of the previous and next two frames are both identified as type A, and the abnormal type of the abnormal audio signal of the current frame is identified as type B, then the abnormal type of the abnormal audio signal of the current frame is smoothed to type A through voting.

[0111] Different from the prior art, this embodiment can predict the approximate abnormal type of the abnormal audio signal by inputting the acquired acoustic features of the abnormal audio signal to be detected into a multi-classifier for preliminary detection, and then establish a more targeted binary classifier based on the results of the preliminary detection, and input the acoustic features into the binary classifier for classification, which can greatly reduce the interference of similar sounds on the recognition of abnormal audio signals, thereby improving the accuracy of abnormal sound detection. In addition, by combining the features of multiple domains for detection, the abnormal audio signal can also be analyzed from multiple angles, thereby further improving the detection and recognition rate of abnormal sounds, and because the detected acoustic features are multidimensional, the robustness of the detection can also be enhanced.

[0112] Correspondingly, the present application provides an abnormal sound classification system.

[0113] Specifically, see Figure 6 , Figure 6 Schematic diagram of the structure of an abnormal sound classification system of the present application. Figure 6 As shown, the abnormal sound classification system 60 includes a feature acquisition module 61 , a first classification module 62 and a second classification module 63 .

[0114] The feature acquisition module 61 is used to acquire the acoustic features of the abnormal audio signal to be detected.

[0115] The first classification module 62 is used to input the acoustic features into a multi-classifier, and use the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features.

[0116] The second classification module 63 is used to detect the acoustic features using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods in response to the difference between the two maximum likelihoods being less than a set value, so as to obtain the abnormal type of the abnormal audio signal.

[0117] For the specific classification process, please refer to the relevant text descriptions in steps S11 to S13, S21 to S23, S31 to S32, S41 to S44 and S51 to S52, which will not be repeated here.

[0118] Different from the prior art, this embodiment acquires the acoustic features of the abnormal audio signal to be detected through the feature acquisition module 61, and inputs the acoustic features into the first classification module 62 for preliminary detection, which can predict the approximate abnormal type of the abnormal audio signal, and then establish a more targeted binary classifier based on the results of the preliminary detection, and input the acoustic features into the second classification module 63 for classification, which can greatly reduce the interference of similar sounds on the recognition of abnormal audio signals, thereby improving the accuracy of abnormal sound detection. In addition, by combining the features of multiple domains for detection, the abnormal audio signal can also be analyzed from multiple angles, thereby further improving the detection and recognition rate of the abnormal sound classification system 60, and because the detected acoustic features are multidimensional, the robustness of the abnormal sound classification system 60 can also be enhanced.

[0119] Correspondingly, the present application provides an abnormal sound classification device.

[0120] Specifically, see Figure 7 , Figure 7 Schematic diagram of the structure of an abnormal sound classification device of the present application. Figure 7 As shown, the abnormal sound classification device 70 includes a memory 71 and a processor 72 coupled to each other.

[0121] In this embodiment, the memory 71 is used to store program data, which, when executed, can implement the steps in any of the abnormal sound classification methods described above; the processor 72 is used to execute the program instructions stored in the memory 71 to implement the steps in any of the abnormal sound classification methods described above.

[0122] Specifically, the processor 72 is used to control itself and the memory 71 to implement the steps in any of the above-mentioned abnormal sound classification method embodiments. The processor 72 can also be called a CPU (Central Processing Unit). The processor 72 may be an integrated circuit chip with signal processing capabilities. The processor 72 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 72 can be implemented by multiple integrated circuit chips.

[0123] Different from the prior art, the processor 72 inputs the acquired acoustic features of the abnormal audio signal to be detected into a multi-classifier for preliminary detection, which can predict the approximate abnormal type of the abnormal audio signal, and then establishes a more targeted binary classifier based on the results of the preliminary detection, and inputs the acoustic features into the binary classifier for classification, which can greatly reduce the interference of similar sounds on the recognition of abnormal audio signals, thereby improving the accuracy of abnormal sound detection. In addition, by combining the features of multiple domains for detection, the abnormal audio signal can also be analyzed from multiple angles, thereby further improving the detection and recognition rate of abnormal sounds, and because the detected acoustic features are multidimensional, the robustness of the detection can also be enhanced.

[0124] Correspondingly, the present application provides a computer-readable storage medium.

[0125] See also Figure 8 , Figure 8 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application.

[0126] The computer-readable storage medium 80 includes a computer program 801 stored on the computer-readable storage medium 80 . When the computer program 801 is executed by the processor, the steps in any one of the abnormal sound classification methods described above are implemented.

[0127] Specifically, if the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 80. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium 80, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 80 includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0128] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0130] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0131] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0132] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for classifying abnormal sounds, It is characterized in that include: Acquiring acoustic features of an abnormal audio signal to be detected; Inputting the acoustic features into a multi-classifier, and using the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features; In response to the difference between the two maximum likelihoods being less than a set value, the acoustic features are detected using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the abnormal audio signal, including: in response to the difference between the two maximum likelihoods being less than the set value, the two abnormal categories corresponding to the two maximum likelihoods are extracted to establish the binary classifier; wherein the binary classifier includes a Gaussian kernel function; the acoustic features of the abnormal audio signal are input into the binary classifier, the statistical features with a long distance are extracted based on clustering using the Gaussian kernel function, and the acoustic features are classified based on the statistical features to obtain the abnormal type of the abnormal audio signal; The step of detecting the acoustic feature using binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the abnormal audio signal in response to the difference between the two maximum likelihoods being less than a set value also includes: determining the abnormal type of the abnormal audio signal using the abnormal types of a set number of consecutive frames of abnormal audio signals; the step of determining the abnormal type of the abnormal audio signal using the abnormal types of a set number of consecutive frames of abnormal audio signals includes: counting the number of frames of abnormal audio signals belonging to each abnormal type; and determining the abnormal type including the largest number of frames as the abnormal type of the abnormal audio signal.

2. The abnormal sound classification method according to claim 1, It is characterized in that The step of acquiring the acoustic features of the abnormal audio signal to be detected includes: Acquiring the abnormal audio signal to be detected; Performing multi-dimensional feature extraction on the abnormal audio signal to obtain the multi-dimensional acoustic features of the abnormal audio signal; The multi-dimensional acoustic features include at least two of time domain features, frequency domain features and cepstrum domain features.

3. The abnormal sound classification method according to claim 1 or 2, It is characterized in that The step of acquiring the acoustic features of the abnormal audio signal to be detected includes: Acquire the abnormal audio signal, and perform frame processing on the abnormal audio signal; Extract features of each frame of abnormal audio signal to obtain acoustic features of each frame of abnormal audio signal; The step of inputting the acoustic features into a multi-classifier and using the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features comprises: Inputting the acoustic features of the abnormal audio signals of each frame into the multi-classifier to obtain the likelihoods of the abnormal audio signals of each frame and the various abnormal categories; In response to the difference between the two maximum likelihoods being less than a set value, the step of detecting the acoustic feature using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain the abnormal type of the abnormal audio signal includes: In response to a difference between two maximum likelihoods of the current frame being less than a set value, the current frame is detected using two binary classifiers of two abnormal categories corresponding to the two maximum likelihoods to obtain an abnormal type of the current frame; The abnormal type of the abnormal audio signal is determined by using the abnormal types of a set number of consecutive frames of abnormal audio signals.

4. The abnormal sound classification method according to claim 3, It is characterized in that The step of extracting features from each frame of abnormal audio signal to obtain acoustic features of each frame of abnormal audio signal includes: Performing windowing processing on each frame of abnormal audio signal; The time domain signal after the windowing process is transformed into a frequency domain signal by using a fast Fourier transform, and then a signal energy spectrum is obtained based on the frequency domain signal; Using a filter to perform bandpass filtering on the signal energy spectrum, and taking the logarithm of the output of the filter to transform it into a logarithmic energy spectrum domain; The logarithmic energy spectrum domain is transformed into the cepstrum domain by using discrete cosine transform to obtain independent feature vectors in each dimension.

5. The abnormal sound classification method according to claim 1, It is characterized in that Before the step of acquiring the acoustic features of the abnormal audio signal to be detected, the method further includes: Acquire an audio signal, and calculate the short-time energy of the audio signal; Comparing the short-time energy with a set energy threshold; In response to the short-term energy of the audio signal being greater than the set energy threshold, the audio signal is determined to be the abnormal audio signal to be detected.

6. A system for classifying abnormal sounds, It is characterized in that include: A feature acquisition module, used to acquire the acoustic features of the abnormal audio signal to be detected; A first classification module, configured to input the acoustic features into a multi-classifier, and use the multi-classifier to determine the likelihood of the abnormal audio signal and various abnormal categories based on the acoustic features; The second classification module is used for detecting the acoustic features using two abnormal categories corresponding to the two maximum likelihoods in response to the difference between the two maximum likelihoods being less than a set value, so as to obtain the abnormal type of the abnormal audio signal. The second classification module is specifically used for extracting the two abnormal categories corresponding to the two maximum likelihoods to establish the binary classifier in response to the difference between the two maximum likelihoods being less than the set value; wherein the binary classifier includes a Gaussian kernel function; the acoustic features of the abnormal audio signal are input into the binary classifier, the statistical features with a long distance are extracted based on the clustering method using the Gaussian kernel function, and the acoustic features are classified based on the statistical features to obtain the abnormal type of the abnormal audio signal; the second classification module is also used for determining the abnormal type of the abnormal audio signal using the abnormal types of a continuous set amount of multiple frames of abnormal audio signals; the step of determining the abnormal type of the abnormal audio signal using the abnormal types of a continuous set amount of multiple frames of abnormal audio signals comprises: counting the number of frames of abnormal audio signals belonging to each abnormal type; and determining the abnormal type including the largest number of frames as the abnormal type of the abnormal audio signal.

7. An abnormal sound classification device, It is characterized in that include: A memory for storing program data, wherein the stored program data, when executed, implements the steps of the abnormal sound classification method according to any one of claims 1 to 5; A processor is used to execute the program instructions stored in the memory to implement the steps in the abnormal sound classification method according to any one of claims 1 to 5.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the abnormal sound classification method according to any one of claims 1 to 5 are implemented.

Citation Information

Patent Citations

  • Short-time and long-time feature modeling fusion-based environmental sound recognition method and device

    CN105654944A

  • Method for identifying states of high-voltage switch cabinets on basis of electric discharge sound

    CN106782505A