Asymmetric pooling and mel-spectrogram based method for recognizing heterophonic sounds in strong noise environment
By combining asymmetric pooling with Mel spectrum analysis, the problem of reduced frequency resolution and confusion in headphone noise recognition under strong noise conditions in traditional convolutional neural networks is solved, achieving high-precision headphone noise classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-04-24
- Publication Date
- 2026-07-14
AI Technical Summary
In the case of headphone noise recognition in a noisy environment, the traditional two-dimensional convolutional neural network has a low recognition rate due to the decrease in frequency resolution and confusion of noise types caused by symmetric pooling.
A method based on asymmetric pooling and Mel spectrum is adopted. The Mel spectrum is processed by logarithmic scaling and an asymmetric pooling structure is introduced into the convolutional neural network to preserve the frequency dimension resolution and extract the time-domain features of long-series signals.
This improved the classification accuracy of the headphone detection system in environments with strong background noise, reduced misjudgments of different sound types, and enhanced the system's stability and computational efficiency.
Smart Images
Figure CN122395535A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of headphone acoustic testing and audio signal processing technology, specifically relating to a method for identifying abnormal sounds in a strong noise environment based on asymmetric pooling and Mel spectrum. Background Technology
[0002] In modern headphone manufacturing, acoustic noise detection of core sound-generating structures such as voice coils, diaphragms, and housings, as well as their assembly status, is a crucial means of achieving headphone quality control. However, real production line testing environments are often accompanied by strong environmental background noise, and the subtle acoustic defects of the target headphones are frequently masked by extremely low signal-to-noise ratios, such as -6dB of high-intensity background noise. While existing technologies have achieved some progress in addressing these issues, the following key technical bottlenecks remain:
[0003] (1) Traditional two-dimensional convolutional neural networks (2D-CNN) usually directly apply the symmetric pooling structure from the field of image recognition when processing audio. However, since the vertical axis of the two-dimensional spectrum of audio represents frequency and the horizontal axis represents time, symmetric pooling will cause irreversible information compression and loss in the frequency dimension.
[0004] (2) For specific types of headphone noises with similar acoustic characteristics, such as voice coil bumps and diaphragm noises, the acoustic differences are mainly reflected in extremely narrow specific harmonic frequency bands. Under strong test noise interference, traditional symmetrical pooling operations tend to remove these weak frequency differences, leading to cross-judgment phenomena.
[0005] Therefore, there is a need for a neural network recognition method that can adapt to low signal-to-noise ratio testing environments and effectively protect high-precision frequency resolution when extracting audio features. Summary of the Invention
[0006] To address the issues of reduced frequency resolution due to symmetric pooling and low recognition rates caused by confusion between different noise types under strong test noise in existing technologies, this invention proposes a headphone noise recognition method based on asymmetric pooling and Mel spectrum analysis in noisy environments. This invention enhances weak signal features under low signal-to-noise ratio conditions by extracting high-dimensional Mel spectra and introducing logarithmic scaling. Simultaneously, an asymmetric pooling structure is designed in the convolutional neural network, enabling the network to extract temporal features of long-series signals while compressing the time dimension, thus preserving the original frequency resolution.
[0007] The technical solution adopted in this invention is as follows:
[0008] An abnormal sound identification method based on asymmetric pooling and Mel spectrum in a high-noise environment is proposed for accurately classifying various types of headphone acoustic defects in a high-background-noise environment. The method assumes that the audio acquired by the test microphone is mono or stereo raw waveform data, and includes the following steps:
[0009] S1. Raw Audio Signal Preprocessing and Mel Spectrum Processing: The raw one-dimensional audio signal from the target earphone is read and fused into a two-channel waveform, converting it into a mono waveform signal. A short-time Fourier transform is performed on the discrete one-dimensional time-domain signal, and a Hanning window is applied to reduce spectral leakage. Subsequently, a Mel filter bank is applied for feature mapping, followed by logarithmic nonlinear scaling, and then global minima normalization is used to obtain the Mel spectrum. Mel frequency. The nonlinear mapping relationship between the frequency f and the physical frequency is expressed as follows:
[0010] ,
[0011] To enhance the faint abnormal sound characteristics of headphones in strong ambient noise, the extracted Mel spectrum was analyzed. Apply logarithmic scaling:
[0012] ,
[0013] in The characteristic matrix is logarithmically scaled. The initial Mel spectrum, For time frame indexing, The Mel frequency band index is then used. Subsequently, the feature matrix is mapped to the [0, 1] interval using the minimax normalization method, and the normalized two-dimensional matrix is uniformly interpolated and scaled to 128. A 128-dimensional single-channel input feature map is used as the input to a convolutional neural network.
[0014] S2. Construct a convolutional neural network based on an asymmetric pooling structure, and input the preprocessed single-channel feature matrix into it for feature extraction. The convolutional neural network sequentially includes an input layer, a first convolutional layer, a first asymmetric pooling layer, a second convolutional layer, a second asymmetric pooling layer, a third convolutional layer, a symmetric pooling layer, a global average pooling layer, and a fully connected classification layer. To solve the frequency resolution blurring problem caused by traditional pooling, this invention uses an asymmetric max pooling layer with a size of 1×2 and a stride of 1×2 in the pre-pooling layer of the network, such as... Figure 4 A comparison diagram of the principles of asymmetric and symmetric pooling is shown. Let the feature map input to this pooling layer be X, and its dimension be H. W, and the output feature map is Y. Asymmetric pooling operation performs sliding compression only along the time axis width W, while remaining unchanged along the frequency axis height H. Its mathematical expression is:
[0015]
[0016] in, For frequency axis index, This is the time axis index after pooling. This asymmetric pooling operation only performs downsampling along the time dimension by half, while the resolution of the frequency dimension remains unchanged. By continuously stacking multiple layers of convolutional blocks with asymmetric pooling, the network extracts complex features while fully preserving the frequency mapping of the vertical axis.
[0017] After asymmetric feature extraction through a first convolutional layer, a first asymmetric pooling layer, a second convolutional layer, a second asymmetric pooling layer, and a third convolutional layer, the deep features are spatially reduced using a global average pooling layer before being input into a fully connected layer. For example... Figure 3 As shown in the overall network design and data flow diagram, after the asymmetric feature extraction stage, a size of 2 is applied to the deep feature map. A standard max-pooling layer of size 2 is used for final joint dimensionality reduction, followed by a global average pooling layer to replace the traditional fully connected layer, thereby reducing the number of model parameters and suppressing overfitting in noisy environments. Finally, the feature logic value output by the fully connected layer is defined as... The total number of categories is The confidence scores for each category are output through a fully connected layer, and the probability distribution expression for the target device category is obtained using the Softmax function.
[0018] ,
[0019] in The feature logic value of the k-th category output by the fully connected layer. Input signal Category The probability of.
[0020] S3. Compare and judge the calculated probabilities to achieve the final classification of abnormal sound in the headphones and determine the probabilities of each defect category obtained in the previous step. Perform numerical sorting; select the category label corresponding to the maximum probability and use it as the final headphone acoustic defect classification output.
[0021] The beneficial effects of this invention are as follows: By fusing the log-Mel spectrum of audio signals with the asymmetric pooling mechanism in a two-dimensional convolutional neural network, this invention improves the classification accuracy of the headphone detection system under strong background noise or production line interference environments. Employing a lossless pooling model along the frequency axis and combining it with the acoustic characteristics of weak harmonics optimizes feature extraction data and effectively reduces the impact of strong environmental noise. Global optimization of the feature matrix through asymmetric pooling fully utilizes local information in the high-dimensional frequency space, not only improving overall classification accuracy but also effectively reducing the impact of misjudgments caused by similar acoustic features in a single headphone defect category. Furthermore, this invention enhances system stability while improving accuracy, enabling it to maintain high-precision classification in complex production line environments. Overall, this invention improves the computational efficiency of the acoustic detection system by optimizing the network pooling process and algorithm implementation, making it adaptable to the batch concurrent testing needs of large-scale headphone production lines. Attached Figure Description
[0022] Figure 1 This is a flowchart illustrating the overall method design of the present invention.
[0023] Figure 2 This is a flowchart of the time-frequency mapping and preprocessing process of the present invention.
[0024] Figure 3 This is a schematic diagram of the overall network design and data flow of the present invention.
[0025] Figure 4 This is a comparison diagram of the principles of asymmetric pooling and symmetric pooling in this invention.
[0026] Figure 5 This is a monitoring graph of training progress and loss curves in an embodiment of the present invention.
[0027] Figure 6 This invention provides a multi-category device classification confusion matrix diagram based on the asymmetric pooling neural network output by the present invention. Detailed Implementation
[0028] The specific technical solutions of the present invention will be described below with reference to the accompanying drawings and embodiments.
[0029] This invention provides a method for classifying abnormal sounds in headphones under strong noise conditions based on asymmetric pooling and Mel-frequency spectrum. For example... Figure 1 As shown in the overall method design flowchart, this method utilizes a convolutional neural network to extract features from the playback audio of various headphone defects. Combined with two-dimensional matrix data from the time-frequency plot, it employs asymmetric max pooling and logarithmic scaling for global optimization. By fusing this information, it can not only achieve accurate classification in harsh environments or production line interference, but also protect and optimize classification accuracy through feature dimension optimization, thereby improving the stability and real-time performance of the detection system.
[0030] The specific implementation steps of this invention are summarized as follows:
[0031] Step 1: Environmental parameter settings: Set parameters such as audio sampling rate, window function length, number of megapixel bands, number of abnormal sound defect categories, and background noise interference ratio.
[0032] Step 2: Preprocessing and Data Generation: such as Figure 2 As shown in the time-frequency mapping and preprocessing flowchart, the original audio signal undergoes two-channel fusion to simulate real-world acoustic signal conversion. Log-Mel bandgap mapping generates high-resolution feature-normalized matrix data.
[0033] Step 3: Network Construction and Feature Estimation: A pre-convolutional module with asymmetric pooling layers is used to perform temporal compression and frequency domain preservation of the input features. Then, deep convolutions are combined with global average pooling layers for high-dimensional feature extraction.
[0034] Step 4: Optimization and Evaluation: The relative probabilities of multiple feature spaces are fused using a fully connected layer and a Softmax function to calculate the classification results of the final aligned labels, and the results are evaluated using the validation set accuracy and confusion matrix.
[0035] Example:
[0036] This embodiment uses the Matlab simulation platform for the experiment.
[0037] The specific implementation method of this invention involves acoustic monitoring of a simulated test production line exhibiting headphone acoustic defects. It utilizes the Mel-frequency spectrum of audio with background noise and an asymmetric pooling structure for multiple iterative training processes, combined with a Softmax function to optimize the global output. In this example, environmental parameters are first set, including basic audio attributes and the number of defect categories. It is assumed that four types of headphone noise defects exist within the same production line test scenario: diaphragm damage, voice coil contact, foreign object noise, and loose housing. All headphone test sounds are masked by the production line background white noise with a signal-to-noise ratio of -6dB, meaning the noise energy is four times the effective signal.
[0038] Step 1: Setting Environmental Parameters
[0039] Basic sampling configuration: Set the audio sampling rate to 16000Hz.
[0040] Short-time Fourier transform: Set the Hanning window length to 1024 and the frame overlap length to 512.
[0041] Noise and resolution settings: Define the number of Mel filter banks to 128 to generate highly accurate low-frequency and high-frequency resolution, and uniformly adjust the output image matrix to 128. 128 dimensions.
[0042] Step 2: Generating Feature Data. For each audio waveform, the system generates a time-frequency matrix using a preprocessing function. The data transformation is achieved through the following steps: First, the original two-channel audio signal is averaged along each channel to convert it into a mono audio signal, and the original energy spectrum is calculated using a discrete Fourier transform. Next, based on the eigenvalues, a logarithmic function is applied to generate high-contrast feature data. Finally, after minima normalization, a standardized matrix suitable for network input is obtained.
[0043] Step 3: Network Model and Feature Extraction. A convolutional neural network is constructed to process time-frequency features. Both the first and second convolutional modules of the network structure use 16 to 32 3D features. The convolution kernel of size 3 extracts local texture. The core change lies in the pooling layer setting, which uses an asymmetric sliding window of [1, 2]. This change effectively achieves half-sampling along the time axis, thus enabling lossless transfer of 128-dimensional vertical frequency features and avoiding the limitations of traditional 2-dimensional convolution kernels. 2. High-frequency detail erasure caused by symmetric pooling.
[0044] Step 4: Align the output with the classification. The dimensionality reduction result based on the global average pooling layer is input into the fully connected layer. After centering and probability calculation, the predicted features are aligned with the true labels to obtain the final classification network model.
[0045] The accuracy of the above scheme was evaluated: the localization and classification accuracy was assessed by calculating the confusion matrix between the predicted and ground truth labels. The accuracy was calculated by dividing the number of correctly predicted samples by the total number of samples. Based on the experimental scenario described above, the experimental results of different models under strong test noise were statistically analyzed. The results show that the overall classification accuracy of the model is improved after introducing the asymmetric pooling mechanism. Figure 5 Training progress chart and Figure 6 As shown in the multi-category device classification confusion matrix, specific abnormal sound defects with extremely similar acoustic features were successfully separated by protecting the frequency axis resolution. The overall verification set accuracy of the system reached 92.23% through calculation, which demonstrates the effectiveness of the method of the present invention.
Claims
1. A method for identifying abnormal sounds in strong noise environments based on asymmetric pooling and Mel-frequency spectrum, used to classify various types of headphone acoustic defects, characterized in that... The audio acquired by the test microphone is set to be mono or stereo raw waveform data. The identification method includes the following steps: S1. Raw audio signal preprocessing and Mel spectrogram processing, specifically including: reading the raw one-dimensional audio signal from the target earphone and performing dual-channel fusion, then converting it into a mono waveform signal; performing a short-time Fourier transform on the discrete one-dimensional time-domain signal and applying a Hanning window to reduce spectral leakage; subsequently applying a Mel filter bank for logarithmic nonlinear scaling of the feature map, and then obtaining the Mel spectrogram and Mel frequency through global maximum and minimum normalization. The nonlinear mapping relationship between the frequency f and the physical frequency is expressed as follows: ； Mel spectrum Perform logarithmic scaling: ， in The characteristic matrix is logarithmically scaled. The initial Mel spectrum, For time frame indexing, The Mel frequency band index is then used; subsequently, the feature matrix is mapped to the [0, 1] interval using the minimax normalization method, and the normalized two-dimensional matrix is uniformly interpolated and scaled to 128. 128-dimensional single-channel input feature map; S2. Construct a convolutional neural network based on asymmetric pooling, and input the preprocessed single-channel input feature map into it for feature extraction. The convolutional neural network sequentially includes an input layer, a first convolutional layer, a first asymmetric pooling layer, a second convolutional layer, a second asymmetric pooling layer, a third convolutional layer, a symmetric pooling layer, a global average pooling layer, and a fully connected classification layer. The first and second asymmetric pooling layers are 1×2 asymmetric pooling layers with a stride of 1×2. The data processing method is as follows: define the feature map input to the asymmetric pooling layer as X, with a dimension of H. W, the output feature map is Y. The asymmetric pooling operation only performs sliding compression on the time axis width W, while remaining unchanged on the frequency axis height H. The mathematical expression is: ， in, For frequency axis index, The time axis index is the result of pooling. The asymmetric pooling operation only performs downsampling along the time dimension by half, while the resolution of the frequency dimension remains unchanged. After asymmetric feature extraction is performed sequentially through the first convolutional layer, the first asymmetric pooling layer, the second convolutional layer, the second asymmetric pooling layer, and the third convolutional layer, a size of 2 is applied to the deep feature map. A standard max pooling layer of size 2 is used for final joint dimensionality reduction, followed by a global average pooling layer. This significantly reduces the number of model parameters and suppresses overfitting in noisy environments. The feature logic value output by the fully connected layer is defined as... The total number of categories is The confidence scores for each category are output through a fully connected layer, and the probability distribution expression for the target device category is obtained using the Softmax function. ， in The feature logic value of the k-th category output by the fully connected layer. Input signal Category The probability of; S3. Compare and judge the calculated probabilities to achieve the final classification of abnormal sound in the headphones and the probability of each defect category. Perform numerical sorting; select the category label corresponding to the maximum probability and use it as the final headphone acoustic defect classification output.