Sound event classification method for reducing reverberation based on frequency band division

Abstract

The invention provides a sound event classification method for reducing reverberation based on frequency band division, which can effectively reduce the influence of background noise such as reverberation and improve the accuracy of sound classification and recognition. A to-be-processed audio signal is converted from a time domain to a frequency domain to obtain a to-be-processed spectrum signal, the to-be-processed spectrum signal is subjected to frequency division ratio control of frequency band bandwidth and reverberation reduction of a selected frequency band to obtain a reverberation-reduced spectrum signal, and the reverberation-reduced spectrum signal is combined with other signals to obtain a finally-output spectrum signal after reverberation reduction. Converting the frequency spectrum signal into a time domain signal and outputting the time domain signal; a training set after data enhancement is formed based on a final output time domain signal, and a sound event classification model is trained, so that the characterization capability of the sound event classification model to features is indirectly enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of sound event classification, and in particular to a sound event classification method based on frequency band division to reduce reverberation. Background Art

[0002] In the natural environment, vision, sound, etc. are important senses for people to perceive the surrounding environment. Through many senses, people can obtain rich information about the surrounding environment. At present, with the development of deep learning technology, people's research on machine vision is becoming more and more perfect, and corresponding sound-related events have been studied in various directions. Sound event classification refers to the classification of sounds generated in daily environments and surroundings. Sound event classification research is the key to developing intelligent applications in the fields of monitoring systems, robot navigation systems, and multimedia searches for audio. Many current models can easily identify voiceprint information and semantic information, such as the patent shown in application number CN201910539745.8, which discloses a method for classifying urban sound events. However, it is found in actual applications that when identifying events in complex scenes in reality, the accuracy of sound event classification is often not high due to other background noises such as reverberation and echo. Summary of the invention

[0003] In order to solve the problem that the accuracy of sound event classification in the prior art is low due to reverberation, echo and other background noise when the sound classification method is applied in complex scenes, the present invention provides a sound event classification method based on frequency band division to reduce reverberation, which can effectively reduce the influence of background noise such as reverberation and improve the accuracy of sound classification and recognition.

[0004] The technical solution of the present invention is as follows: a sound event classification method based on frequency band reduction of reverberation, comprising the following steps:

[0005] S1: Collecting original audio sample information, performing preprocessing operations on it, and obtaining a preprocessed audio signal to be processed;

[0006] It is characterized in that it also includes the following steps:

[0007] S2: The audio signal to be processed is transformed from the time domain S(t) to the frequency domain through short-time Fourier transform to obtain a spectrum signal to be processed S(t,ω), where t is time and ω is frequency;

[0008] S3: Perform frequency component analysis on the spectrum signal to be processed S(t,ω) to obtain the highest frequency ω in the signal h and the lowest frequency ω l , and then get the input signal bandwidth B corresponding to the spectrum signal S(t,ω) to be processed i ;

[0009] Bi =ω h -ω l ;

[0010] S4: Based on the acquisition scenario of the original audio sample information, set the frequency division ratio r, select the frequency band signal and control the output bandwidth B of the selected frequency band o ;

[0011] B o =B i *r;

[0012] S5: Based on the output bandwidth B0 of the selected frequency band, the spectrum signal to be processed S(t,ω) is sent to a Butterworth filter, and based on the Butterworth frequency domain response, a Butterworth amplitude function |H(ω)| in the frequency domain and a selected spectrum signal S1(t,ω) corresponding to B0 are obtained;

[0013]

[0014] Where N is the order, ω c is the cut-off frequency;

[0015] S6: using the weighted prediction error algorithm WPE to reduce the reverberation of the selected spectrum signal S1 (t, ω), and obtaining a spectrum signal output S2 (t, ω) after the reverberation is reduced;

[0016]

[0017] in, It is estimated by the prediction filter in WPE, H is the conjugate transpose operator, D is the prediction delay, and D divides the reverberation environment into early reverberation and late reverberation. This is the late reverberation part;

[0018] S7: subtract the selected spectrum signal S1(t,ω) from the spectrum signal to be processed S(t,ω) to obtain another signal S3(t,ω);

[0019] Combine S2(t,ω) with other signals S3(t,ω) to obtain the final output spectrum signal S4(t,ω) after reducing reverberation, and then use the short-time Fourier inverse transform to convert the signal S4(t,ω) from the frequency domain to the time domain to obtain S4(t);

[0020]

[0021] S8: Based on the deep residual shrinkage network DRSN, a frequency domain residual shrinkage network F-DRSN is constructed;

[0022] The frequency domain residual shrinkage network F-DRSN comprises: a convolution layer, a residual shrinkage unit, a global average pooling and a fully connected layer connected in sequence; wherein the number of the residual shrinkage units is greater than 1; each of the residual shrinkage units comprises a frequency domain self-correction algorithm;

[0023] S9: constructing a sound event classification model based on the frequency domain residual shrinkage network F-DRSN; training the sound event classification model to obtain the trained sound event classification model;

[0024] S10: Acquire sound signal data to be detected, extract acoustic features from the sound signal data to be detected, and obtain acoustic features to be identified;

[0025] The acoustic features to be identified are sent to the trained sound event classification model to obtain the final classification result.

[0026] It is further characterized by:

[0027] In step S5, the order N and the cutoff frequency ω c The calculation method is:

[0028]

[0029] Among them, δ1 is the maximum attenuation in the passband, and δ2 is the minimum attenuation in the stopband;

[0030] ω p is the passband cutoff frequency, ω s is the stopband cutoff frequency, ω p and ω s The calculation method is as follows:

[0031]

[0032] In the formula, when the filtered frequency band is a high frequency band, ω p Recorded as ω s Recorded as When the filtered frequency band is a low frequency band, ω p Recorded as ω s Recorded as

[0033] ω h is the highest frequency of the spectrum signal S(t,ω) to be processed, ω l is the lowest frequency of the spectrum signal S(t,ω) to be processed; B o is the output bandwidth of the selected frequency band;

[0034] and They are respectively the bandwidth of the high-frequency transition band and the bandwidth of the low-frequency transition band;

[0035] The residual shrinkage unit includes: a frequency domain self-correction module, two convolutional layers and a soft thresholding module, and an attention module connected in sequence. The output of the last convolutional layer is sent to the soft thresholding module, and is also sent to the attention module to extract typical features in the channel, and then sent to the soft thresholding module to remove noise;

[0036] The operations in the frequency domain self-correction module include:

[0037] a1: compress the frequency domain dimension of the input feature x through MaxPooling;

[0038] a2: Use two convolutional layers to extract features from the compressed results;

[0039] a3: UpSampling is performed to restore the size to the same as the initial input feature size, and the extracted frequency domain features are summed with the original feature map to obtain the output feature x1;

[0040] a4: compress x1 into a one-dimensional vector through global average pooling, and obtain the recalibrated result through the fully connected layer, which is recorded as x2;

[0041] a5: multiply the result of the original feature after two convolutional layers by x2 to get the final output x3;

[0042] In step S9, training the sound classification model specifically includes the following steps:

[0043] b1: Select the data set and use the original audio data samples to divide the data set into training set and test set;

[0044] b2: Calculate and obtain the time domain signal S4(t) after reducing reverberation in the frequency bands corresponding to the training set;

[0045] b3: Fully mix the original time domain signal S(t) corresponding to the training set and the time domain signal S4(t) after frequency band reduction of reverberation to form a training set after data enhancement;

[0046] b4: extracting acoustic features from the enhanced training set to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficients. The three acoustic features are recorded as: training acoustic feature set;

[0047] b5: sending the training acoustic feature set to the sound event classification model to train the sound event classification model, and obtaining the trained sound event classification model for training;

[0048] b6: extracting acoustic features from the data of the test set respectively to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficient, and the three acoustic features are recorded as: acoustic feature set for verification;

[0049] b7: sending the verification acoustic feature set into the trained sound event classification model, and judging the model training result according to the classification result;

[0050] In step S10, the method for extracting acoustic features from the sound signal data to be detected is as follows:

[0051] c1: Confirm the classification result obtained by sending the verification acoustic feature set into the trained sound event classification model, and find the acoustic feature with the highest classification accuracy, which is recorded as the recognition acoustic feature;

[0052] c2: after extracting the acoustic features for identification from the sound signal data to be detected, record them as: acoustic features to be identified;

[0053] In step S1, the original audio sample information includes: audio duration and sampling frequency;

[0054] The preprocessing operation includes: pre-emphasis, framing and windowing.

[0055] The present application provides a sound event classification method based on frequency band division and reverberation reduction, which processes the audio signal to be processed and then converts it from the time domain S(t) to the frequency domain to obtain the spectrum signal to be processed S(t, ω), controls the frequency band bandwidth of the spectrum signal to be processed by the frequency division ratio r, reduces the reverberation in the selected frequency band, and obtains the spectrum signal output S2(t, ω) after the reverberation is reduced; then S2(t, ω) is merged with other signals S3(t, ω) to obtain the final output spectrum signal S4(t, ω) after the reverberation is reduced, and then converts it into a time domain signal S4(t); based on the final output time domain signal S4(t) forming a data enhanced training set, the sound event classification model is trained to obtain a trained sound event classification model; the method uses the frequency band division and reverberation reduction method to calculate the final output signal after the reverberation is reduced, effectively reduces the background noise such as the reverberation in the original audio, minimizes the sound distortion and improves the clarity of the overall speech, thereby indirectly enhancing the feature representation capability of the sound event classification model, and further improving the classification accuracy of the sound event classification model. At the same time, the present application also introduces the frequency domain self-correction algorithm into the DRSN model to construct a sound event classification model. The frequency domain self-correction module increases the attention to the frequency domain characteristics of the input sound data, fully extracts the frequency domain characteristics of the sound data, and combines the reduced final output time domain signal S4(t) obtained after processing by the frequency band reverberation reduction method, thereby further improving the classification accuracy of the sound event classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a schematic diagram of the principle of frequency band reduction of reverberation in this application;

[0057] Figure 2 A schematic diagram of the network structure of the sound event classification model in this application;

[0058] Figure 3 Schematic diagram of the network structure of the frequency domain self-correction module in the present invention. DETAILED DESCRIPTION

[0059] like Figure 1 As shown, the present application includes a sound event classification method based on frequency band division to reduce reverberation, which includes the following steps:

[0060] S1: Collect original audio sample information, perform preprocessing on it, improve signal quality, and obtain a preprocessed audio signal to be processed.

[0061] The original audio sample information includes: audio duration and sampling frequency. The preprocessing operations include: pre-emphasis, framing and windowing; among them, pre-emphasis enhances the high-frequency components in the speech signal, framing processes the signal into a stable signal, and windowing is used to reduce the spectrum leakage problem caused by the signal stage.

[0062] S2: The audio signal to be processed is transformed from the time domain S(t) to the frequency domain through short-time Fourier transform to obtain the spectrum signal to be processed S(t,ω), where t is time and ω is frequency.

[0063] S3: Perform frequency component analysis on the spectrum signal S(t,ω) to be processed, analyze the various frequency components in the signal, and obtain the highest frequency ω in the signal h and the lowest frequency ω l , and then get the input signal bandwidth B corresponding to the spectrum signal S(t,ω) to be processed i ;

[0064] B i =ω h -ω l .

[0065] S4: Based on the acquisition scenario of the original audio sample information, set the frequency division ratio r, select the frequency band signal and control the output bandwidth B of the selected frequency band o ;

[0066] B o =B i *r.

[0067] In specific implementation, the frequency division ratio is set according to the sound collection scenario, obtained through historical experience, or through experimental data testing.

[0068] S5: Based on the output bandwidth B0 of the selected frequency band, the spectrum signal S(t,ω) to be processed is sent to the Butterworth filter, and based on the Butterworth frequency domain response, the Butterworth amplitude function |H(ω)| in the frequency domain and the selected spectrum signal S1(t,ω) corresponding to B0 are obtained;

[0069]

[0070] Where N is the order, ω c is the cut-off frequency.

[0071] Depending on the original audio collection scene, the spectrum signal obtained after the Butterworth filter processing may be a high-frequency signal or a low-frequency signal. For example, the sound data collected in the indoor environment includes mostly low-frequency signals, while the signals collected in the outdoor environment in the city have a higher probability of being classified as high-frequency signals. This method is based on the setting of the frequency division ratio r, and uses the Butterworth filter to achieve the frequency division effect of the spectrum signal to be processed. Figure 1 In, S h1 (t,ω) indicates that the signal obtained after Butterworth filter filtering is a high-frequency signal, S l1 (t,ω) indicates that the signal obtained after Butterworth filter filtering is a low-frequency signal. h3 (t,ω) and S l3 (t,ω) represent the h1 (t,ω) and S l1 (t,ω) Other high-frequency signals and other low-frequency signals calculated.

[0072] In step S5, based on the prior art, it is known that the order N and the cutoff frequency ω c The calculation method is:

[0073]

[0074] Among them, δ1 is the maximum attenuation in the passband, and δ2 is the minimum attenuation in the stopband. In practical applications, δ1 and δ2 are set based on experiments; δ1 is generally small to ensure the signal quality in the passband, and δ2 is generally large to suppress interference signals.

[0075] ω p is the passband cutoff frequency, ω s is the stopband cutoff frequency; ω p to s The frequency band between is the transition band, ω p and ω s The calculation method is as follows:

[0076]

[0077] In the formula, when the filtered frequency band is a high frequency band, ω p Recorded as ω s Recorded as When the filtered frequency band is a low frequency band, ω p Recorded as ω s Recorded as ω h is the highest frequency of the spectrum signal S(t,ω) to be processed, ω l is the lowest frequency of the spectrum signal S(t,ω) to be processed; B o is the output bandwidth of the selected frequency band; where, and They are the bandwidth of the high-frequency transition band and the bandwidth of the low-frequency transition band respectively. and The value of is selected according to the actual scene needs; for example, in this embodiment, in the high-frequency band filtering scenario, it is expected that the transition band range is larger, and Set to 1000hz, or in the low-frequency filtering scenario, Set to 300 Hz; when determining the passband cutoff frequency of the high or low frequency band ω p and stopband cutoff frequency ω s Then, the cutoff frequency ω can be obtained. c and order N.

[0078] S6: using the weighted prediction error algorithm WPE to reduce the reverberation of the selected spectrum signal S1(t,ω), and obtaining the spectrum signal output S2(t,ω) after the reverberation is reduced;

[0079]

[0080] in, It is estimated by the prediction filter in WPE, H is the conjugate transpose operator, D is the prediction delay, and D divides the reverberation environment into early reverberation and late reverberation. This is the late reverberation part. Figure 1 In, S h2 (t,ω) and S l2 (t, ω) represent the spectrum signal output after reducing reverberation calculated based on the high-frequency band signal and the low-frequency band signal respectively.

[0081] Speech signals usually consist of three parts, namely direct signal, early reverberation, and late reverberation. Since early reverberation has a positive effect on speech clarity, but late reverberation reduces speech clarity, the emphasis is often placed on eliminating late reverberation. Therefore, the S1(t,ω) signal is reduced by the weighted prediction error method (WPE), and the spectrum signal output S2(t,ω) of the S1(t,ω) signal with reduced high frequencies or reduced low frequencies can be obtained.

[0082] S7: subtract the selected spectrum signal S1(t,ω) from the spectrum signal to be processed S(t,ω) to obtain other signals S3(t,ω); combine the spectrum signal S2(t,ω) after reducing reverberation with other signals S3(t,ω) to obtain the final output signal S4(t,ω) after reducing reverberation, and then use the short-time inverse Fourier transform to convert the signal from the frequency domain to the time domain to obtain the time domain output S4(t); This method effectively reduces the background noise such as high-frequency or low-frequency reverberation in the original audio, has little damage to the sound quality, and at the same time enhances the clarity of the speech;

[0083]

[0084] That is, the obtained time domain output signal S4(t) is the data obtained by removing the reverberation data in the specified frequency band from the time domain signal S(t) to be processed. Figure 1 In, S h4 (t) and S l4 (t) represents the final output signal calculated based on the high frequency band signal and the final output signal calculated based on the low frequency band signal.

[0085] S8: Based on the deep residual shrinkage network DRSN, the frequency domain self-correction algorithm is introduced to construct the frequency domain residual shrinkage network F-DRSN;

[0086] like Figure 2 As shown in the figure, the frequency domain residual shrinkage network F-DRSN includes: a convolution layer Conv, a residual shrinkage unit Residual Shrinkage Block, a global average pooling Global Average Pooling and a fully connected layer FC connected in sequence. The fully connected layer FC also includes a Dropout function and a softmax function (not marked in the figure) connected in sequence; the input is the acoustic feature Sound Feature of the sound signal data to be detected, and the output is the sound classification result SoundClassification.

[0087] In the frequency domain residual shrinkage network F-DRSN, the first convolution layer Conv is Conv2D, the size of the convolution kernel is 3*3, and the number of convolution kernels is 128, which performs preliminary feature extraction. The fully connected layer FC uses global average pooling to pool the multidimensional array after the convolution operation into a one-dimensional array, and connects to the Dropout function to prevent overfitting. Finally, the sound event classification model outputs the classification prediction through the softmax function. The softmax calculation formula is:

[0088]

[0089] Si represents the softmax function, i is a positive integer, and Zi is the output value of the fully connected layer.

[0090] The number of residual shrinkage units included in the frequency domain residual shrinkage network F-DRSN is greater than 1. Figure 2 In the illustrated embodiment, three residual shrinkage units are included, each of which is followed by a BN+Relu layer. The present application introduces a frequency domain self-correction algorithm in the residual shrinkage unit to enhance the model's ability to collect frequency domain information of the input feature Sound Feature.

[0091] The residual shrinkage unit includes: a frequency domain self-correction module, two convolutional layers and a soft thresholding module connected in sequence, and an attention module. The output of the last convolutional layer is sent to the soft thresholding module, and then sent to the attention module to extract typical features in the channel, and then sent to the soft thresholding module to remove noise. In the attention module, Abs represents absolute value, GAP represents global average pooling, FC represents fully connected layer, Maxpooling represents maximum pooling, Sigmoid and Softmax are both activation functions.

[0092] like Figure 3 As shown, the operations in the frequency domain self-correction module include:

[0093] a1: compress the frequency domain dimension of the input feature x through MaxPooling;

[0094] a2: Use two convolutional layers to extract features from the compressed results, effectively extracting frequency domain features;

[0095] a3: UpSampling is performed to restore the size to the same as the initial input feature size, and the extracted frequency domain features are summed with the original feature map to obtain the output feature x1;

[0096] a4: compress x1 into a one-dimensional vector through global average pooling, and obtain the recalibrated result through the fully connected layer, which is recorded as x2;

[0097] a5: multiply the result of the original feature after two convolutional layers by x2 to get the final output x3;

[0098] In this application, the frequency domain self-correction algorithm is introduced into the deep residual shrinkage network DRSN to form F-DRSN, which enhances the classification model's attention to the frequency domain; the frequency domain self-correction can more effectively extract the contextual information in the frequency domain by compressing the frequency dimension, and utilizes the residual structure and channel attention to aggregate the effective information in the shallow features and deep features, and extract the typical features in the channel according to the channel attention map.

[0099] The structure of the convolution layer in the frequency domain self-correction module is: Conv2D+BatchNormalization+Relu, the size of the convolution kernel is 3*3, and the number of convolution kernels is 128. Soft thresholding is to put the soft threshold denoising method into the deep learning network, so that the network adaptively adjusts the threshold of potential noise in the input signal, reduces the more obvious noise signal to close to 0, and successfully eliminates most of the noise features in the input signal.

[0100] The soft threshold function can be expressed as follows:

[0101]

[0102] Among them, |x| is the coefficient of wavelet transform, and t is the pre-selected threshold. The threshold is quantified using the fixed threshold method, which is defined as: t = σ2log(N), where N is the number of signal samples and σ is the standard deviation of the noise.

[0103] S9: Construct a sound event classification model based on the frequency domain residual shrinkage network F-DRSN; train the sound event classification model to obtain a trained sound event classification model.

[0104] In step S9, the sound classification model is trained, which specifically includes the following steps:

[0105] b1: Select the data set and use the original audio data samples to divide the data set into training set and test set;

[0106] b2: Get the original time domain signal S(t) of the training set data, and calculate the time domain signal S4(t) after frequency division and reverberation reduction;

[0107] b3: Mix the original time domain signal S(t) of the training set and the final time domain signal S4(t) after reducing reverberation to form a training set after data enhancement;

[0108] b4: extracting acoustic features from the enhanced training set to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficients. The three acoustic features are recorded as: training acoustic feature set;

[0109] b5: sending the training acoustic feature set to the sound event classification model, training the sound event classification model, and obtaining the trained sound event classification model for further training;

[0110] b6: Extract acoustic features from the test set data respectively to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficients. The three acoustic features are recorded as: acoustic feature set for verification;

[0111] b7: Send the acoustic feature set for verification into the trained sound event classification model, and judge the model training results based on the classification results. If the effect is not good, re-train; if the effect meets expectations, stop training.

[0112] S10: Acquire the sound signal data to be detected, extract acoustic features from the sound signal data to be detected, and obtain acoustic features to be identified; send the acoustic features to be identified into the trained sound event classification model to obtain the final classification result.

[0113] In step S10, the method for extracting acoustic features from the sound signal data to be detected is as follows:

[0114] c1: Confirm the classification results obtained by sending the verification acoustic feature set into the trained sound event classification model, and find the acoustic feature with the highest classification accuracy, which is recorded as the recognition acoustic feature;

[0115] For example, during training, it is found that the acoustic feature with the highest classification accuracy for low-frequency data is the log-mel spectrum, so the log-mel spectrum is recorded as the acoustic feature for recognition;

[0116] c2: After extracting the acoustic features for identification from the sound signal data to be detected, it is recorded as: acoustic features to be identified;

[0117] In actual classification, the logarithmic Mel-spectrogram features are extracted from the sound signal data to be detected and sent to the trained sound event classification model for classification.

[0118] In the present method, the frequency division ratio r is set in a targeted manner according to different types of sound signal data to be detected, and the corresponding frequency division band data features are extracted to eliminate reverberation noise, and then the final time domain signal S4(t) after reducing the reverberation is used to train the sound event classification model, so as to ensure that the trained classification model is particularly suitable for the sound signal data to be detected; at the same time, during the training, the acoustic features with the highest classification accuracy corresponding to the sound signal data to be detected in the sound event classification model are observed and recorded as the acoustic features for recognition, and then in the actual test, the acoustic features for recognition corresponding to the sound data to be detected are used for classification, so as to further improve the classification accuracy.

[0119] In order to confirm the performance of the sound event classification method based on frequency band reduction of reverberation in the technical solution of the present invention, in the experimental environment of Window11 system, RTX3080Ti (12G video memory), and 32G memory, Keras+TensorFlow is used as the deep learning framework, and the urban sound event classification standard dataset UrbanSound8K is adopted, wherein: fold1-9 is used as the training set, the number of training set samples is 7895, and the number of training sets after data enhancement is 15790; the test set is the wav audio file in fold10, and the number of samples is 838; the following experiments are carried out respectively:

[0120] (1) Performance comparison of different network models;

[0121] (2) Comparison of classification accuracy of reverberation reduction models in different frequency bands;

[0122] (3) Reduce the optimal frequency division ratio of the high-frequency reverberation model;

[0123] (4) Classification accuracy of different data augmentation methods.

[0124] The CNN model, the common residual network ResNet-18, and the model F-DRSN in this application are used as classification models, and Log-Mel, Gamma, and GFCC are used as features. A sound event classification experiment is carried out on the Urbansound8K dataset, and the accuracy of the three models is compared. The specific results are shown in Table 1.

[0125] Table 1 Performance comparison of different network models

[0126] Network Model Log-Mel Gamma GFCC CNN 83.37％ 81.58％ 80.38％ ResNet-18 84.81％ 83.49％ 83.25％ F-DRSN 85.17％ 84.57％ 83.85％

[0127] As shown in Table 1, compared with the convolutional neural network CNN, the patented network F-DRSN has improved 1.8%, 2.99%, and 3.47% in the three features respectively; compared with the common residual network ResNet-18, the patented network F-DRSN has improved 0.36%, 1.08%, and 0.6% in the three features respectively. In summary, the representation ability of the F-DRSN model is stronger than that of CNN and ResNet-18, so its classification accuracy has been significantly improved.

[0128] The sound event classification model (F-DRSN) in this application is used as the classification model, and Log-Mel, Gamma, and GFCC are used as features. A sound event classification experiment is performed on the Urbansound8K dataset, and the accuracy is compared. The specific results are shown in Tables 2, 3, and 4.

[0129] Table 2 Comparison of classification accuracy of reverberation reduction models in different frequency bands (r = 0.3)

[0130] Frequency band Log-Mel Gamma GFCC No data augmentation 85.17％ 84.57％ 83.85％ Low frequency band 85.65％ 87.32％ 85.89％ High frequency band 87.80％ 87.68％ 86.12％ Full frequency band 86.72％ 86.36％ 86.01％

[0131] As shown in Table 2, (1) compared with the experiment without data enhancement, from the perspective of each frequency band, the classification accuracy of the network model is improved to varying degrees when reducing reverberation in the low frequency band, high frequency band, and full frequency band. Among them, the performance of reducing reverberation in the high frequency band is the best, with accuracy rates of 87.80%, 87.68%, and 86.12% respectively. At the same time, in the high frequency band, the network has better results when using Log-Mel features as input than when using other features as input. (2) In terms of using various features as input, most models perform better on Log-Mel features, except for the low frequency band, where Gamma and GFCC features have higher accuracy than Log-Mel features, by 1.67% and 0.24% respectively.

[0132] Table 3 Optimal crossover ratio for reducing reverberation model in high frequency band

[0133] Frequency division ratio r 0.1 0.2 0.3 0.35 0.4 0.5 0.6 0.7 0.9 Accuracy 85.41％ 86.48％ 87.80％ 89.12％ 86.00％ 87.92％ 85.77％ 85.65％ 85.41％

[0134] As shown in Table 3, (1) when r is from 0.1 to 0.35, the effect of reducing high-frequency reverberation is better and the distortion to the original sound is less, so the accuracy gradually increases; (2) Since it is inevitable to cause a certain degree of distortion to the original sound while reducing high-frequency reverberation, r is from 0.35 to 0.4, the degree of reducing high-frequency reverberation gradually increases, and the distortion ratio also increases. When r is from 0.35 to 0.4, the advantage of reducing high-frequency reverberation and enhancing speech intelligibility is dominant, and the accuracy has recovered a little. (3) When r is from 0.5 to 0.9, the larger r is, the higher the proportion of original audio distortion is, resulting in a reduction in effective audio information, resulting in a decrease in the accuracy of the model. (4) The experiment verified that when the frequency division ratio is 0.35, the accuracy of the model is the highest, indicating that this parameter is the optimal parameter.

[0135] Table 4 Classification accuracy of different data enhancement methods

[0136] Data augmentation methods Log-Mel Gamma GFCC Baseline 85.17％ 84.57％ 83.85％ Dynamic Range Compression 87.44％ 87.32％ 86.84％ Time Stretch 88.16％ 86.01％ 86.96％ Pitch Shift 87.20％ 87.32％ 85.65％ Reduce high frequency reverberation 89.12％ 88.28％ 87.80％

[0137] It can be seen from Table 4 that compared with the traditional data enhancement method, the data enhancement method for reducing high-frequency reverberation performs best among the three acoustic features. The classification accuracy of the Log-Mel feature with the best performance in reducing high-frequency reverberation is 0.96%, 0.96%, and 0.84% ​​higher than the best performing time stretching, pitch shifting or dynamic range compression, and time stretching among the three acoustic features after data enhancement methods, respectively.

[0138] In summary, based on the classification accuracy comparison in Table 1, it can be seen that the feature expression ability of F-DRSN is better than that of common networks CNN and ResNet-18;

[0139] Based on the classification accuracy data in Table 2, it can be seen that reducing high-frequency reverberation performs better in the frequency band reverberation reduction method. The sound classification model in the basic embodiment is a classification model for high-frequency band data; because high-frequency band sounds account for the majority of urban sounds, the experimental data and the collection environment of the original sound data are matched;

[0140] Based on the classification accuracy in Table 3, the optimal frequency division ratio r=0.35 can be obtained, that is, when the sound classification model in this embodiment is used to classify the high-frequency band spectrum signal to be processed, the frequency division ratio r is set to 0.35;

[0141] Based on Table 4, it can be seen that compared with the traditional data enhancement method, the data enhancement method that reduces high-frequency reverberation significantly improves the network representation ability.

[0142] The sound event classification method provided in the present application divides the audio data into frequency bands to reduce the data enhancement of reverberation before processing the audio data, takes the mixed audio of the enhanced audio and the original audio as input, extracts the acoustic features and inputs them into the model. The method can effectively reduce background noise such as high-frequency reverberation or low-frequency reverberation in the original audio, with little damage to the sound quality, and at the same time enhances the clarity of the speech, thereby improving the representation ability of the model and the classification accuracy of the model, so that the accuracy of the sound event classification model has been improved to a certain extent.

Claims

1. A sound event classification method based on frequency band reduction of reverberation, comprising the following steps: S1: Collecting original audio sample information, performing preprocessing operations on it, and obtaining a preprocessed audio signal to be processed; It is characterized in that it also includes the following steps: S2: The audio signal to be processed is transformed from the time domain S(t) to the frequency domain through short-time Fourier transform to obtain a spectrum signal to be processed S(t,ω), where t is time and ω is frequency; S3: Perform frequency component analysis on the spectrum signal to be processed S(t,ω) to obtain the highest frequency ω in the signal h and the lowest frequency ω l , and then get the input signal bandwidth B corresponding to the spectrum signal S(t,ω) to be processed i ; B i =ω h -oh l ; S4: Based on the acquisition scenario of the original audio sample information, set the frequency division ratio r, select the frequency band signal and control the output bandwidth B of the selected frequency band o ; B o =B i *r; S5: Based on the output bandwidth B0 of the selected frequency band, the spectrum signal to be processed S(t,ω) is sent to a Butterworth filter, and based on the Butterworth frequency domain response, a Butterworth amplitude function |H(ω)| in the frequency domain and a selected spectrum signal S1(t,ω) corresponding to B0 are obtained; Where N is the order, ω c is the cut-off frequency; S6: using the weighted prediction error algorithm WPE to reduce the reverberation of the selected spectrum signal S1 (t, ω), and obtaining a spectrum signal output S2 (t, ω) after the reverberation is reduced; in, It is estimated by the prediction filter in WPE, H is the conjugate transpose operator, D is the prediction delay, and D divides the reverberation environment into early reverberation and late reverberation. This is the late reverberation part; S7: subtract the selected spectrum signal S1(t,ω) from the spectrum signal to be processed S(t,ω) to obtain another signal S3(t,ω); Combine S2(t,ω) with other signals S3(t,ω) to obtain the final output spectrum signal S4(t,ω) after reducing reverberation, and then use the short-time Fourier inverse transform to convert the signal S4(t,ω) from the frequency domain to the time domain to obtain S4(t); S8: Based on the deep residual shrinkage network DRSN, a frequency domain residual shrinkage network F-DRSN is constructed; The frequency domain residual shrinkage network F-DRSN comprises: a convolution layer, a residual shrinkage unit, a global average pooling and a fully connected layer connected in sequence; wherein the number of the residual shrinkage units is greater than 1; each of the residual shrinkage units comprises a frequency domain self-correction algorithm; S9: constructing a sound event classification model based on the frequency domain residual shrinkage network F-DRSN; training the sound event classification model to obtain the trained sound event classification model; S10: Acquire sound signal data to be detected, extract acoustic features from the sound signal data to be detected, and obtain acoustic features to be identified; The acoustic features to be identified are sent to the trained sound event classification model to obtain the final classification result.

2. According to claim 1, a sound event classification method based on frequency band reduction of reverberation is characterized by: In step S5, the order N and the cutoff frequency ω c The calculation method is: Among them, δ1 is the maximum attenuation in the passband, and δ2 is the minimum attenuation in the stopband; ω p is the passband cutoff frequency, ω s is the stopband cutoff frequency, ω p and ω s The calculation method is as follows: In the formula, when the filtered frequency band is a high frequency band, ω p Recorded as ω s Recorded as When the filtered frequency band is a low frequency band, ω p Recorded as ω s Recorded as ω h is the highest frequency of the spectrum signal S(t,ω) to be processed, ω l is the lowest frequency of the spectrum signal S(t,ω) to be processed; B o is the output bandwidth of the selected frequency band; and They are respectively the bandwidth of the high-frequency transition band and the bandwidth of the low-frequency transition band.

3. The sound event classification method based on frequency band division and reverberation reduction according to claim 1, characterized in that: The residual shrinkage unit includes: a frequency domain self-correction module, two convolutional layers and a soft thresholding module, and an attention module connected in sequence. The output of the last convolutional layer is sent to the soft thresholding module, and is also sent to the attention module to extract typical features in the channel, and then sent to the soft thresholding module to remove noise.

4. The sound event classification method based on frequency band division and reverberation reduction according to claim 3, characterized in that: The operations in the frequency domain self-correction module include: a1: compress the frequency domain dimension of the input feature x through MaxPooling; a2: Use two convolutional layers to extract features from the compressed results; a3: UpSampling is performed to restore the size to the same as the initial input feature size, and the extracted frequency domain features are summed with the original feature map to obtain the output feature x1; a4: compress x1 into a one-dimensional vector through global average pooling, and obtain the recalibrated result through the fully connected layer, which is recorded as x2; a5: Multiply the result of the original feature after two convolutional layers by x2 to get the final output x3.

5. The sound event classification method based on frequency band division and reverberation reduction according to claim 1, characterized in that: In step S9, training the sound classification model specifically includes the following steps: b1: Select the data set and use the original audio data samples to divide the data set into training set and test set; b2: Calculate and obtain the time domain signal S4(t) after reducing reverberation in the frequency bands corresponding to the training set; b3: Fully mix the original time domain signal S(t) corresponding to the training set and the time domain signal S4(t) after frequency band reduction of reverberation to form a training set after data enhancement; b4: extracting acoustic features from the enhanced training set to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficients. The three acoustic features are recorded as: training acoustic feature set; b5: sending the training acoustic feature set to the sound event classification model to train the sound event classification model, and obtaining the trained sound event classification model for training; b6: extracting acoustic features from the data of the test set respectively to obtain: logarithmic Mel spectrum, gammatone spectrum and gammatone frequency cepstrum coefficient, and the three acoustic features are recorded as: acoustic feature set for verification; b7: Send the verification acoustic feature set to the trained sound event classification model, and judge the model training results according to the classification results.

6. The sound event classification method based on frequency band division and reverberation reduction according to claim 1, characterized in that: In step S10, the method for extracting acoustic features from the sound signal data to be detected is as follows: c1: Confirm the classification result obtained by sending the verification acoustic feature set into the trained sound event classification model, and find the acoustic feature with the highest classification accuracy, which is recorded as the recognition acoustic feature; c2: After extracting the acoustic features for identification from the sound signal data to be detected, it is recorded as: acoustic features to be identified.

7. The method for classifying sound events based on frequency band division and reverberation reduction according to claim 1, characterized in that: In step S1, the original audio sample information includes: audio duration and sampling frequency; The preprocessing operation includes: pre-emphasis, framing and windowing.

Images