A Noise-Aware Annoyance Prediction Method Based on Convolutional-Recurrent Neural Networks
By using a convolutional recurrent neural network-based method, the problems of low efficiency and insufficient accuracy in noise comfort evaluation are solved, achieving high-precision prediction of noise perceived annoyance, especially performing well in the high perceived annoyance range, and providing stable objective evaluation indicators.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING UNIV
- Filing Date
- 2026-01-21
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies suffer from low efficiency, insufficient accuracy, and limited generalization ability in noise comfort assessment. In particular, they are not accurate enough in predicting the subjective perceived annoyance of band-limited noise and are difficult to adapt to noise samples of different frequency bands and bandwidths.
By employing a convolutional recurrent neural network-based approach, a deep neural network model is constructed through preprocessing of the noise signal and extraction of time-frequency features. The model is trained using a training sample set and the predicted value is corrected through a linear correction function, thereby achieving end-to-end prediction from noise signal to subjectively perceived annoyance level.
It achieves high-precision prediction of perceived annoyance level across the entire range of 0 to 10, and maintains a low absolute error and an average relative error of about 1% to 2% on high-perceived annoyance noise samples. It can replace part of the subjective listening and evaluation work and provide stable objective indicators.
Smart Images
Figure CN122090882A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of noise control, specifically a noise perception annoyance prediction method based on a convolutional recurrent neural network. Background Technology
[0002] In the fields of noise control, including indoor environments, vehicles, air conditioning, and household appliances, the evaluation of noise comfort currently relies on two methods:
[0003] (1) Subjective listening test: Subjects are organized to score a series of noise samples in a standard listening environment and give a subjective perceived annoyance level of 0 to 10. This method is time-consuming and labor-intensive. The state of the subjects and the experimental environment can easily introduce bias. Moreover, whenever the product structure, working conditions or noise conditions change, the listening test needs to be reorganized, resulting in very low evaluation efficiency.
[0004] (2) Empirical evaluation based on simple acoustic indicators: In engineering, single indicators such as sound pressure level, A-weighted sound level or loudness, or simple linear combinations, are often used to approximate the subjective perceived annoyance of noise. These methods mainly reflect the energy level and do not adequately consider the shape of the noise spectrum, modulation characteristics, band-limiting characteristics, etc. The prediction accuracy of subjective perceived annoyance of noise in different frequency bands and complex band-limited noise is limited, especially in the high perceived annoyance range, where large deviations are prone to occur.
[0005] Existing technologies generally suffer from the following problems:
[0006] (1) Low evaluation efficiency: It relies heavily on offline subjective listening and evaluation, has high labor costs, and cannot quickly process a large number of noisy samples;
[0007] (2) Insufficient prediction accuracy: Simple acoustic indicators are difficult to capture the complex relationship between time-frequency structure and human ear perception, and the prediction of perceived annoyance of band-limited noise is not accurate enough.
[0008] (3) Limited generalization ability: Models based on empirical formulas are often only applicable to specific working conditions or specific products, and are difficult to adapt to noise samples of different frequency bands and bandwidths.
[0009] Therefore, it is necessary to propose a model and method that can automatically predict the subjective perceived level of annoyance directly from noise signals, so as to improve evaluation efficiency and reduce the workload of subjective listening and evaluation while ensuring prediction accuracy. Summary of the Invention
[0010] The purpose of this invention is to provide a noise-perceived annoyance prediction method based on a convolutional recurrent neural network, comprising the following steps:
[0011] Step 1) Obtain noise samples and corresponding perceived annoyance scores, and preprocess the noise samples to generate a preprocessed monophonic noise signal sample set;
[0012] Step 2) Extract features from each mono noise signal sample to generate a time-frequency image;
[0013] Step 3) Combine the time-frequency images in chronological order to generate a feature sequence; use the feature sequence as input and the perceived annoyance score as output to construct a training sample set and a validation sample set;
[0014] Step 4) Construct a deep neural network model and train the deep neural network model using the training sample set to obtain a noise perception annoyance prediction model;
[0015] Step 5) Use the validation sample set to fit the univariate linear relationship between the actual perceived annoyance level and the original network prediction value to obtain a linear correction function;
[0016] Step 6) Obtain the predicted value of noise perception annoyance using the noise perception annoyance prediction model;
[0017] Step 7) Use a linear correction function to correct the noise perception annoyance prediction value to obtain the final noise perception annoyance prediction structure.
[0018] Furthermore, in step 1), the preprocessing steps include filtering, unifying the sampling rate and channels, and unifying the duration.
[0019] Furthermore, in step 1), the perceived annoyance score is obtained through a questionnaire survey.
[0020] Furthermore, in step 2), the feature extraction step for each monophonic noise signal sample includes:
[0021] Step 2.1) Set the window length, frame shift, and sampling frequency, and perform a short-time Fourier transform on the mono noise signal samples;
[0022] Among them, the The short-time Fourier transform results of the frames are shown below:
[0023] (1)
[0024] In the formula, This is a monophonic noise signal sample. For frame shift; For window length; For analysis window;
[0025] Step 2.2) Calculate the power spectrum or amplitude spectrum of the short-time Fourier transform result, i.e.:
[0026] (2)
[0027] In the formula, Power spectrum; Amplitude spectrum;
[0028] Step 2.3) Calculate the frequency index for each Mel filter. Weight at point ,Right now:
[0029] (3)
[0030] In the formula, ;
[0031] Step 2.4) Process the short-time Fourier transform result using a Mel filter to obtain the log-Mel feature;
[0032] The output power of the b-th mel filter in the m-th frame is shown below:
[0033] (4)
[0034] in, A constant to prevent logarithmic divergence; For the first The output power of each Mel filter;
[0035] The log-Mel characteristics are as follows:
[0036] (5)
[0037] In the formula, It is a log-Mel feature;
[0038] Step 2.5) Arrange all log-Mel features in chronological order to obtain the log-Mel spectrum;
[0039] Step 2.6) By cropping or copying the edge frequency bands, the frequencies of the log-Mel spectrogram are unified to the preset number of frequency bands, and the time dimension is cropped or padded according to a fixed number of frames to generate a time-frequency image.
[0040] Furthermore, the deep neural network model includes a two-dimensional convolution module, an LSTM, and a regression output module;
[0041] Two-dimensional convolutional modules are used to extract local time-frequency features;
[0042] The LSTM is used to capture the long and short-term time dependencies of local time-frequency features;
[0043] The regression output module takes the output of LSTM as input data and the perceived annoyance score as output.
[0044] Furthermore, the output of the 2D convolution module is shown below:
[0045] (6)
[0046] In the formula, It is a non-linear activation function; As weight; It is a local time-frequency feature.
[0047] Furthermore, the two-dimensional convolution module includes a first convolution block and a second convolution block with the same structure; the number of channels and the number of convolution kernels in the second convolution block are greater than the number of channels and the number of convolution kernels in the first convolution block.
[0048] Both the first convolutional block and the second convolutional block include a two-dimensional convolutional layer, a batch normalization layer, a non-linear activation layer, and an average pooling layer;
[0049] The output of the two-dimensional convolutional layer As shown below:
[0050] (7)
[0051] In the formula, For the first The weights of each convolutional kernel; For bias; For input;
[0052] The output of the batch normalization layer is shown below:
[0053]
[0054] In the formula, For channel The mean and variance; These are the scaling and translation parameters; This is the standardized convolution output; This is the output of the batch normalization layer; It is a constant used to prevent division by zero;
[0055] Output of nonlinear activation layer As shown below:
[0056] (10)
[0057] The output of the average pooling layer is shown below:
[0058] (11)
[0059] In the formula, This is the output of the average pooling layer.
[0060] Furthermore, the output of the regression output module is shown below:
[0061] (13)
[0062] (15)
[0063] In the formula, This is the output of the LSTM; This is the output of the first fully connected layer; The mask vector is obtained by sampling from the Bernoulli distribution. This indicates element-wise multiplication; This is the predicted value of perceived annoyance for this noise sample.
[0064] Furthermore, the loss function used when training a deep neural network model... As shown below:
[0065] (16)
[0066] in For the first The actual perceived level of annoyance of the sample. These are the model's predicted values.
[0067] Furthermore, the linear correction function is shown below:
[0068] (17)
[0069] (18)
[0070]
[0071] In the formula, This is the final prediction result for perceived annoyance level. This is the correction factor.
[0072] It is worth noting that this invention provides a noise-perceived annoyance prediction method based on a convolutional-recurrent neural network. For band-limited noise samples, the original noise signal is first uniformly preprocessed and time-frequency features are extracted to obtain a fixed-size log-Mel spectrum sequence. Then, feature encoding and temporal modeling are performed through a two-level two-dimensional convolutional network and a long short-term memory network (LSTM). Finally, the perceived annoyance score of 0 to 10 is directly predicted through a regression output layer, realizing end-to-end prediction from noise signal to subjective perceived annoyance.
[0073] The technical effects of this invention are undeniable. This invention can achieve high-precision prediction of perceived annoyance level across the entire range of 0 to 10 points. Especially on high-perceived-annoyance noise samples, it can still maintain a low absolute error and an average relative error of about 1% to 2%. It can effectively replace some of the repetitive subjective listening and evaluation work and provide a stable and reliable objective indicator for noise evaluation. Attached Figure Description
[0074] Figure 1 To verify the distribution map;
[0075] Figure 2 To validate the 2D histogram of the set;
[0076] Figure 3 This is the loss function. Detailed Implementation
[0077] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.
[0078] Example 1:
[0079] See Figures 1-3 A noise-perceived annoyance prediction method based on convolutional recurrent neural networks includes the following steps:
[0080] Step 1) Obtain noise samples and corresponding perceived annoyance scores, and preprocess the noise samples to generate a preprocessed monophonic noise signal sample set;
[0081] Step 2) Extract features from each mono noise signal sample to generate a time-frequency image;
[0082] Step 3) Combine the time-frequency images in chronological order to generate a feature sequence; use the feature sequence as input and the perceived annoyance score as output to construct a training sample set and a validation sample set;
[0083] Step 4) Construct a deep neural network model and train the deep neural network model using the training sample set to obtain a noise perception annoyance prediction model;
[0084] Step 5) Use the validation sample set to fit the univariate linear relationship between the actual perceived annoyance level and the original network prediction value to obtain a linear correction function;
[0085] Step 6) Obtain the predicted value of noise perception annoyance using the noise perception annoyance prediction model;
[0086] Step 7) Use a linear correction function to correct the noise perception annoyance prediction value to obtain the final noise perception annoyance prediction structure.
[0087] Example 2:
[0088] A noise perception annoyance prediction method based on convolutional recurrent neural network, with the same technical content as in Example 1, further, in step 1), the preprocessing steps include filtering, unifying the sampling rate and channels, and unifying the duration.
[0089] Example 3:
[0090] A noise perception annoyance prediction method based on convolutional recurrent neural networks, with the same technical content as any one of embodiments 1-2, further wherein, in step 1), the perceived annoyance score is obtained through a questionnaire survey.
[0091] Example 4:
[0092] A noise perception annoyance prediction method based on a convolutional recurrent neural network, with the same technical content as any one of embodiments 1-3, further comprising, in step 2), the step of feature extraction for each monophonic noise signal sample, including:
[0093] Step 2.1) Set the window length, frame shift, and sampling frequency, and perform a short-time Fourier transform on the mono noise signal samples;
[0094] Among them, the The short-time Fourier transform results of the frames are shown below:
[0095] (1)
[0096] In the formula, This is a monophonic noise signal sample. For frame shift; For window length; For analysis window;
[0097] Step 2.2) Calculate the power spectrum or amplitude spectrum of the short-time Fourier transform result, i.e.:
[0098] (2)
[0099] In the formula, Power spectrum; Amplitude spectrum;
[0100] Step 2.3) Calculate the frequency index for each Mel filter. Weight at point ,Right now:
[0101] (3)
[0102] In the formula, ;
[0103] Step 2.4) Process the short-time Fourier transform result using a Mel filter to obtain the log-Mel feature;
[0104] The output power of the b-th mel filter in the m-th frame is shown below:
[0105] (4)
[0106] in, A constant to prevent logarithmic divergence; For the first The output power of each Mel filter;
[0107] The log-Mel characteristics are as follows:
[0108] (5)
[0109] In the formula, It is a log-Mel feature;
[0110] Step 2.5) Arrange all log-Mel features in chronological order to obtain the log-Mel spectrum;
[0111] Step 2.6) By cropping or copying the edge frequency bands, the frequencies of the log-Mel spectrogram are unified to the preset number of frequency bands, and the time dimension is cropped or padded according to a fixed number of frames to generate a time-frequency image.
[0112] Example 5:
[0113] A noise perception annoyance prediction method based on a convolutional recurrent neural network, with the same technical content as any one of embodiments 1-4, further wherein the deep neural network model includes a two-dimensional convolutional module, an LSTM, and a regression output module;
[0114] Two-dimensional convolutional modules are used to extract local time-frequency features;
[0115] The LSTM is used to capture the long and short-term time dependencies of local time-frequency features;
[0116] The regression output module takes the output of LSTM as input data and the perceived annoyance score as output.
[0117] Example 6:
[0118] A noise-perceived annoyance prediction method based on a convolutional-recurrent neural network, with the same technical content as any one of embodiments 1-5, further wherein the output of the two-dimensional convolutional module is shown below:
[0119] (6)
[0120] In the formula, It is a non-linear activation function; As weight; It is a local time-frequency feature.
[0121] Example 7:
[0122] A noise-perceived annoyance prediction method based on a convolutional recurrent neural network, with the same technical content as any one of embodiments 1-6, further wherein the two-dimensional convolutional module includes a first convolutional block and a second convolutional block with the same structure; the number of channels and the number of convolutional kernels of the second convolutional block are greater than the number of channels and the number of convolutional kernels of the first convolutional block;
[0123] Both the first convolutional block and the second convolutional block include a two-dimensional convolutional layer, a batch normalization layer, a non-linear activation layer, and an average pooling layer;
[0124] The output of the two-dimensional convolutional layer As shown below:
[0125] (7)
[0126] In the formula, For the first The weights of each convolutional kernel; For bias; For input;
[0127] The output of the batch normalization layer is shown below:
[0128]
[0129] In the formula, For channel The mean and variance; These are the scaling and translation parameters; This is the standardized convolution output; This is the output of the batch normalization layer; It is a constant used to prevent division by zero;
[0130] Output of nonlinear activation layer As shown below:
[0131] (10)
[0132] The output of the average pooling layer is shown below:
[0133] (11)
[0134] In the formula, This is the output of the average pooling layer.
[0135] Example 8:
[0136] A noise-perceived annoyance prediction method based on a convolutional recurrent neural network, with the same technical content as any one of embodiments 1-7, further wherein the output of the regression output module is as follows:
[0137] (13)
[0138] (15)
[0139] In the formula, This is the output of the LSTM; This is the output of the first fully connected layer; The mask vector is obtained by sampling from the Bernoulli distribution. This indicates element-wise multiplication; This is the predicted value of perceived annoyance for this noise sample.
[0140] Example 9:
[0141] A noise-perceived annoyance prediction method based on convolutional recurrent neural networks, with the same technical content as any one of embodiments 1-8, further comprising the following steps: when training the deep neural network model, the loss function used... As shown below:
[0142] (16)
[0143] in For the first The actual perceived level of annoyance of the sample. These are the model's predicted values.
[0144] Example 10:
[0145] A noise-perceived annoyance prediction method based on a convolutional recurrent neural network, with the same technical content as any one of embodiments 1-9, further wherein the linear correction function is as follows:
[0146] (17)
[0147] (18)
[0148]
[0149] In the formula, This is the final prediction result for perceived annoyance level. This is the correction factor.
[0150] Example 11:
[0151] A noise-aware annoyance prediction method based on convolutional recurrent neural networks includes the following steps:
[0152] 2. Noise pretreatment
[0153] (1) Input signal type: The noise signal is a band-limited noise sample, that is, the noise in a certain frequency band is retained by the filter, while other frequency components are significantly attenuated or filtered out. Each sample corresponds to a subjective listening evaluation obtained by the perceived annoyance score.
[0154] The noise signal can be a band-limited noise sample obtained through a filter, or broadband noise collected in the actual environment. The current model acquires the noise waveform at a high sampling rate (65536 Hz), then resamples it to a uniform sampling rate (16 kHz), and subsequently extracts the log-Mel spectral features and inputs them into a deep neural network to output the perceived annoyance score of the corresponding noise.
[0155] (2) Unify sampling rate and channels: Resample the input band-limited noise samples and unify the sampling rate to 16kHz. If the sample is a multi-channel recording, average the channels to obtain a mono signal to eliminate the differences caused by different recording methods.
[0156] (3) Segmentation and duration are unified. For excessively short samples, zeros can be padded to ensure that the duration is not less than the preset value; for extremely long samples, they can be divided into several segments with a fixed duration for separate prediction.
[0157] 3. Feature extraction: log-Mel spectrum sequence
[0158] (1) Time-frequency transformation: For the pre-processed monophonic noise signal, short-time analysis is performed with a fixed frame length (e.g., 25 ms) and frame shift (e.g., 10 ms). The Mel spectrum is obtained through short-time Fourier transform and Mel filter bank.
[0159] Short-time Fourier transform formula:
[0160] The preprocessed monophonic noise signal is denoted as... The sampling frequency is .
[0161] Let the window length be (Corresponding to 25 ms), frame shift is (Corresponding to 10 ms), the analysis window is... (e.g., Hanming window). The first The short-time Fourier transform of a frame can be written as:
[0162]
[0163] The power spectrum or amplitude spectrum can be written as:
[0164]
[0165] The subsequent Mel filter bank is in or The above is a weighted sum.
[0166] Mel frequency and Mel filter bank expression
[0167] Mapping from linear frequency to Mel frequency
[0168] For linear frequency (Hz), using the common Mel-transform formula:
[0169]
[0170] At the lowest frequency With the highest frequency Take at equal intervals between them Mel frequency points ( (This represents the number of Mel filters), and then mapping it back to the linear frequency yields three key points for each filter:
[0171]
[0172] Corresponding FFT frequency index
[0173] No. Triangular frequency response of a Mel filter
[0174] No. Mel filters ( In frequency index Weight at point Defined as a piecewise linear triangle:
[0175]
[0176] Mel filter bank output and log-Mel spectrum
[0177] For the Frame, number The output power of each Mel filter is:
[0178]
[0179] Taking the logarithm again yields the log-Mel characteristic:
[0180]
[0181] in To prevent logarithmic divergence, a small constant is used. All frames and all filter outputs are arranged in chronological order, resulting in the log-Mel spectrum of the input to the convolutional-LSTM network.
[0182] (2) Logarithmic compression performs a logarithmic operation on the Mel spectrum to obtain the log-Mel spectrum, which compresses the dynamic range and improves the consistency with human auditory perception.
[0183] (3) Size unification and serialization: By cropping or copying edge bands, the frequency dimension of the Mel spectrum is unified to a preset number of Mel bands (e.g., 64). The time dimension is cropped or padded according to a fixed number of frames (e.g., 398 frames) so that each sample corresponds to a set of time-frequency images of size 64×398×1. These time-frequency images are arranged in chronological order to form a feature sequence, which is used as the input to the subsequent deep neural network. The preset number of Mel bands is determined by a combination of factors such as sampling rate, effective frequency band of interest, required frequency resolution, and model computational complexity. In this embodiment, considering the sampling rate of noise samples and the prediction accuracy requirements, the number of Mel bands is preferably set to 64.
[0184] 4. Model Structure: 2D Convolution + LSTM + Regression Output
[0185] (1) Sequence Input and Folding: The sequence input layer receives the feature sequence consisting of the log-Mel spectrum. The sequence folding layer expands each frame in the time series in the batch dimension so that local time-frequency features can be extracted for each frame using two-dimensional convolution operation.
[0186] The "local time-frequency features" here are essentially the feature values obtained by performing a two-dimensional convolution on a small time-frequency neighborhood in the log-Mel spectrum. This can be expressed mathematically as follows:
[0187] Let the preprocessed log-Mel spectrum be...
[0188] in It is the Mel frequency index ( ), It is a time frame index ( ).
[0189] ②For the first There are two-dimensional convolutional kernels, each with a size of [size missing]. The weight is denoted as
[0190] Bias is .
[0191] ③ The convolution kernel is in the following... The eigenvalues obtained in the local time-frequency neighborhood of the top-left corner can be written as:
[0192]
[0193] in It is a non-linear activation function (e.g., ReLU).
[0194] this It is the mathematical expression for "local time-frequency characteristics":
[0195] "Local": Only used around This small time-frequency neighborhood;
[0196] "Time-Frequency": The horizontal axis represents time frames, and the vertical axis represents frequency band indexes;
[0197] "Features": The output obtained by weighted summation and nonlinear transformation of the convolution kernel. ,along with Slide across the entire spectrogram to obtain a complete feature map.
[0198] If there are multiple channels (e.g., the previous layer has already output), (a feature map), can be written in a more general form:
[0199]
[0200] (2) First convolutional block
[0201] 1) Two-dimensional convolutional layer
[0202] Let the input be ,in
[0203] For the input channel index (log-Mel when there is only 1 channel), ),
[0204] These are the pixel coordinates in the frequency and time directions.
[0205] No. The weights of each convolutional kernel are Size is , bias is The output of the convolutional layer is denoted as... :
[0206]
[0207] 2) Batch Normalization Layer (BN)
[0208] For each channel Convolution output Standardize, then perform linear scaling and translation. Within a small batch, the channels... The mean and variance are respectively The trainable scaling and translation parameters are Then the BN output is :
[0209]
[0210] It is a tiny constant used to prevent division by zero.
[0211] 3) Nonlinear activation layer (ReLU)
[0212] Using a linear rectifier unit, a point-by-point nonlinear transformation is performed on the BN output to obtain the activated features. :
[0213] 4) 2×2 average pooling layer
[0214] Within each channel, perform activation feature processing. Non-overlapping average pooling is used to achieve downsampling. Let the coordinates of the feature map after pooling be... Then the first Each channel pooled output for:
[0215]
[0216] The interlayer relationships of the "first convolutional block" can be summarized as follows:
[0217]
[0218] in It refers to the local time-frequency characteristics of the output of the first convolutional block to the next network module.
[0219] Two-dimensional convolutional layer: Using a 3×3 convolutional kernel and a certain number of convolutional channels, local convolution is performed on the log-Mel spectrum of each frame to extract low-level time-frequency texture features;
[0220] Batch normalization layer: Standardizes the convolutional output, reduces internal covariate bias, and stabilizes training;
[0221] Nonlinear activation layer: typically uses linear rectified units to perform nonlinear transformations on the features;
[0222] Average pooling layer: 2×2 average pooling is used to downsample the feature map, reduce dimensionality and suppress local noise.
[0223] (3) Based on the output features of the first convolutional block, the second convolutional block is then set with a two-dimensional convolutional layer, a batch normalization layer, a nonlinear activation layer, and a flat average pooling layer to extract higher-level time-frequency patterns, such as the shape of the frequency band energy distribution and the modulation structure.
[0224] First convolutional block: 3×3 convolutional kernel, with 32 output channels;
[0225] The second convolutional block has a 3×3 convolutional kernel and 64 output channels.
[0226] The second convolutional block increases the number of convolutional kernels with the same kernel size and pooling method, increasing the number of channels in the feature map from 32 to 64, and extracting higher-level, more abstract local time-frequency features at a lower time-frequency resolution.
[0227] (4) After the sequence unrolling and flattening convolutional blocks, the folded frame features are recombined into a feature sequence in chronological order through the sequence unrolling layer. Then, the flattening layer is used to flatten the two-dimensional feature map into a one-dimensional feature vector, providing input for subsequent temporal modeling.
[0228] (5) The Long Short-Term Memory (LSTM) layer inputs the flattened feature sequence into the LSTM layer. The LSTM unit processes the features of each frame sequentially in the time dimension, and captures the long and short-term temporal dependencies of the entire noise through mechanisms such as input gate, forget gate and output gate. The LSTM layer is set to output only the hidden state vector of the last time step, which serves as the overall representation of the entire band-limited noise in the high-dimensional feature space.
[0229] (6) Regression Output Layer
[0230] ① LSTM output to fully connected + activation: Let the output vector of the LSTM in the last frame be... The output of the first fully connected layer is
[0231]
[0232] After nonlinear activation (such as ReLU) to obtain
[0233]
[0234] ② Dropout regularization:
[0235] in The mask vector is obtained by sampling from the Bernoulli distribution. This indicates element-wise multiplication. If dropout is not enabled, then... .
[0236] ③Final regression output (predicted value of perceived annoyance):
[0237] in This is the predicted value (scalar) of the perceived annoyance level for this noise sample.
[0238] ④ Loss function (mean squared error, MSE): for a batch bar sample The loss is
[0239] in For the first The actual perceived level of annoyance of the sample. These are the model's predicted values.
[0240] The vector output from the LSTM is fed into a fully connected layer and a non-linear activation layer, and regularization can be performed using random dropout to prevent overfitting. Finally, a one-dimensional fully connected layer outputs a real value as the predicted perceived annoyance level for the noise sample. The output layer is a regression layer, and the loss function is the mean squared error (MSE), which is the average of the squares of the difference between the actual perceived annoyance level and the predicted perceived annoyance level.
[0241] 5. Model Training Process
[0242] (1) Training data preparation: 932 samples were obtained from the band-limited noise listening test, each sample corresponding to a subjective perceived annoyance level label of 0 to 10; 746 of them were used as the training set and 186 as the validation set. The training set labels cover the entire score range, with relatively more samples in the low perceived annoyance and high perceived annoyance ranges.
[0243] (2) Label standardization: Before training, the perceived annoyance labels of the training set are standardized by subtracting the mean and then dividing by the standard deviation. The network actually fits the standardized labels, and the predicted output needs to be restored to the original score by destandardization.
[0244] (3) Optimization strategy and regularization: The adaptive gradient descent algorithm is used for training, and the initial learning rate is set to approximately 3 × 10⁻. 4Overfitting is suppressed through L2 regularization and random deactivation, and a gradient threshold is set to prevent gradient explosion. Mini-batch gradient descent is used during training, with each batch containing 4 sample sequences.
[0245] (4) After the linear correction training is completed, the linear correction parameters are obtained by fitting the actual perceived annoyance level to the validation set using the univariate linear relationship between the actual perceived annoyance level and the original network prediction value. In actual use, the original value can be obtained first through network prediction, and then the corrected perceived annoyance level prediction result can be obtained through this linear transformation to reduce systematic bias.
[0246] Fitting on the validation set: for the ... A number of validation samples, recorded as follows:
[0247] The actual perceived level of annoyance is ,
[0248] The original network prediction value is .
[0249] Assume that the two satisfy an approximately linear relationship.
[0250]
[0251] in Let be the correction coefficients to be determined. Solve using the least squares method.
[0252]
[0253] Its closed-form solution is ( , They are respectively , (mean)
[0254]
[0255] The correction formula used in actual application: For any sample, the original network output Its corrected predicted value for perceived annoyance is
[0256] Technical effect
[0257] The model after applying linear correction was evaluated on 186 validation samples, and the following technical results were obtained:
[0258] Overall regression accuracy
[0259] The root mean square error is approximately 0.4331.
[0260] The mean absolute error is approximately 0.2110;
[0261] The correlation coefficient between predicted perceived annoyance and subjective perceived annoyance is approximately 0.985.
[0262]
Claims
1. A noise-perceived annoyance prediction method based on a convolutional recurrent neural network, characterized in that, Includes the following steps: Step 1) Obtain noise samples and corresponding perceived annoyance scores, and preprocess the noise samples to generate a preprocessed monophonic noise signal sample set; Step 2) Extract features from each mono noise signal sample to generate a time-frequency image; Step 3) Combine the time-frequency images in chronological order to generate a feature sequence; Using feature sequences as input and perceived annoyance scores as output, a training sample set and a validation sample set are constructed. Step 4) Construct a deep neural network model and train the deep neural network model using the training sample set to obtain a noise perception annoyance prediction model; Step 5) Use the validation sample set to fit the univariate linear relationship between the actual perceived annoyance level and the original network prediction value to obtain a linear correction function; Step 6) Obtain the predicted value of noise perception annoyance using the noise perception annoyance prediction model; Step 7) Use a linear correction function to correct the noise perception annoyance prediction value to obtain the final noise perception annoyance prediction structure.
2. The noise perception annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, In step 1), the preprocessing steps include filtering, unifying the sampling rate and channels, and unifying the duration.
3. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, In step 1), the perceived annoyance score is obtained through a questionnaire survey.
4. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, Step 2), the feature extraction step for each mono noise signal sample includes: Step 2.1) Set the window length, frame shift, and sampling frequency, and perform a short-time Fourier transform on the mono noise signal samples; Among them, the The short-time Fourier transform results of the frames are shown below: ；（1） In the formula, This is a monophonic noise signal sample. For frame shift; For window length; For analysis window; Step 2.2) Calculate the power spectrum or amplitude spectrum of the short-time Fourier transform result, i.e.: ；（2） In the formula, Power spectrum; Amplitude spectrum; Step 2.3) Calculate the frequency index for each Mel filter. Weight at point ,Right now: ；（3） In the formula, ; Step 2.4) Process the short-time Fourier transform result using a Mel filter to obtain the log-Mel feature; The output power of the b-th mel filter in the m-th frame is shown below: ；（4） in, A constant to prevent logarithmic divergence; For the first The output power of each Mel filter; The log-Mel characteristics are as follows: ；（5） In the formula, It is a log-Mel feature; Step 2.5) Arrange all log-Mel features in chronological order to obtain the log-Mel spectrum; Step 2.6) By cropping or copying the edge frequency bands, the frequencies of the log-Mel spectrogram are unified to the preset number of frequency bands, and the time dimension is cropped or padded according to a fixed number of frames to generate a time-frequency image.
5. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, The deep neural network model includes a two-dimensional convolutional module, an LSTM, and a regression output module; Two-dimensional convolutional modules are used to extract local time-frequency features; The LSTM is used to capture the long and short-term time dependencies of local time-frequency features; The regression output module takes the output of LSTM as input data and the perceived annoyance score as output.
6. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 5, characterized in that, The output of the 2D convolution module is shown below: ；（6） In the formula, It is a non-linear activation function; As weight; It is a local time-frequency feature.
7. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 6, characterized in that, The two-dimensional convolutional module includes a first convolutional block and a second convolutional block with the same structure; the number of channels and the number of convolutional kernels in the second convolutional block are greater than the number of channels and the number of convolutional kernels in the first convolutional block. Both the first convolutional block and the second convolutional block include a two-dimensional convolutional layer, a batch normalization layer, a non-linear activation layer, and an average pooling layer; The output of the two-dimensional convolutional layer As shown below: ；（7） In the formula, For the first The weights of each convolutional kernel; For bias; For input; The output of the batch normalization layer is shown below: In the formula, For channel The mean and variance; These are the scaling and translation parameters; This is the standardized convolution output; This is the output of the batch normalization layer; It is a constant used to prevent division by zero; Output of nonlinear activation layer As shown below: ；（10） The output of the average pooling layer is shown below: ；（11） In the formula, This is the output of the average pooling layer.
8. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 5, characterized in that, The output of the regression output module is shown below: ；（13） ； （15） In the formula, This is the output of the LSTM; This is the output of the first fully connected layer; The mask vector is obtained by sampling from the Bernoulli distribution. This indicates element-wise multiplication; This is the predicted value of perceived annoyance for this noise sample.
9. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, The loss function used when training a deep neural network model As shown below: ；（16） in For the first The actual perceived level of annoyance of the sample. These are the model's predicted values.
10. The noise-perceived annoyance prediction method based on a convolutional-recurrent neural network according to claim 1, characterized in that, The linear correction function is shown below: ；（17） ；（18） In the formula, This is the final prediction result for perceived annoyance level in noise; This is the correction factor.