Pickup performance optimization method and device
Through dynamic convolution kernel selection mechanism, hybrid neural network and generative adversarial network, the noise suppression and context information processing problems of audio signal processing in complex environments are solved, and high-quality audio signal enhancement and noise suppression are achieved, suitable for multimedia and voice recognition field.
Patent Information
- Application Number
- CN202411270698.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-11
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing audio signal processing methods are difficult to effectively separate the target audio signal in complex and changing environments, especially when multiple noise sources exist, and lack sufficient context information processing capabilities, resulting in unsatisfactory noise suppression effect and poor signal quality. Stablize.
Adopt dynamic convolution kernel selection mechanism, improved hybrid neural network structure, cross-channel self-attention mechanism and generative adversarial network, combined with field adaptive technology, adaptively handles diverse audio scenarios, and passes short-time Fourier transform and convolution Neural networks, long and short-term memory networks, bidirectional recurrent neural networks and generative adversarial networks optimize the frequency domain and timing characteristics of audio signals.
It significantly improves the audio processing quality in complex environments, improves noise suppression effect and signal clarity, outputs audio signals with high naturalness and good clarity, and is suitable for fields such as multimedia and voice recognition.
Smart Images

Figure CN119049506A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of audio signal processing, and in particular to a method and device for optimizing sound pickup performance. Background Art
[0002] Against the backdrop of the rapid development of modern information technology, audio signal processing technology has become a key component in the fields of digital communications, entertainment, speech recognition, and smart homes. With the growing demand for high-quality audio experience, how to effectively pick up and enhance audio signals in complex and changing environments has become an urgent problem to be solved. At present, mainstream audio signal processing methods mainly focus on traditional digital signal processing technology and machine learning-based algorithms. Among them, digital signal processing technology usually uses filtering, noise reduction, frequency domain analysis and other means to improve the quality of audio signals, while machine learning algorithms achieve more precise audio enhancement by training models and learning specific audio features.
[0003] Traditional digital signal processing methods, such as the classic Fourier transform and short-time Fourier transform, can convert time domain signals into frequency domain signals, thereby analyzing the frequency components in audio signals. This method has achieved certain success in noise suppression, echo cancellation, etc. However, these methods often rely on prior knowledge of environmental noise and are unable to cope with complex and changing recording environments. Especially when multiple noise sources exist at the same time, traditional filtering technology is difficult to effectively separate the target audio signal, thus affecting the audio quality.
[0004] In recent years, with the development of deep learning, neural network models such as convolutional neural networks (CNN) and recurrent neural networks (RNN) have been widely used in the field of audio signal processing. CNN can effectively extract spatiotemporal features in audio signals, while RNN is good at processing time series data. However, these methods still face some challenges in practical applications. For example, CNN usually uses fixed convolution kernels and cannot adapt to different types of audio signals and noise environments, which limits its performance in diverse audio scenarios. Although RNN can capture dependencies in time series, its structure easily leads to the loss of long-term dependency information, especially when processing long sequence data.
[0005] In response to the above problems, some researchers have proposed improved methods, such as dynamic convolution kernel selection mechanism and hybrid neural network structure. The dynamic convolution kernel selection mechanism can adaptively adjust the size and weight of the convolution kernel according to the input audio features, so that the model can better process audio signals of different frequencies. This method effectively improves the generalization ability of the model in complex environments. In addition, the hybrid structure that combines the long short-term memory network (LSTM) and the gated recurrent unit (GRU) can make full use of the advantages of both, enhance the modeling ability of long-term and short-term dependencies, and reduce the risk of information loss. However, although these improved methods have improved the effect of audio signal processing to a certain extent, there are still some shortcomings in practical applications.
[0006] First, many current methods often fail to effectively distinguish the characteristics of different noise sources during noise suppression and audio enhancement, resulting in less than ideal denoising results. In complex environments, different noise sources may have different spectral characteristics, which requires the model to accurately identify and suppress irrelevant noise while retaining important audio signals. The performance of existing deep learning models in this regard still needs to be improved.
[0007] Secondly, current audio signal processing methods lack sufficient consideration when processing contextual information. Especially in speech recognition and speech enhancement applications, the contextual information of audio signals has a crucial impact on the recognition results. Traditional RNN models are susceptible to information loss and short-sighted dependencies when capturing contextual information. Although bidirectional recurrent neural networks can alleviate this problem to a certain extent, they still have difficulty dealing with long-term dependencies. In addition, although the application of cross-channel self-attention mechanism can enhance the features of specific channels, its weighting method and parameter selection still need to be further optimized.
[0008] In addition, in the process of audio signal generation, although the generative adversarial network can generate realistic audio signals, it still faces the problems of unstable quality and high distortion of the generated signals in the applications of noise suppression and audio enhancement. In particular, for audio signals in different recording environments, the existing GAN model lacks sufficient domain adaptability and cannot effectively cope with the changes in frequency domain features caused by environmental changes. This not only affects the naturalness and clarity of the audio signal, but also limits the practicality of the model.
[0009] Therefore, how to provide a method and device for optimizing sound pickup performance is an urgent problem to be solved by those skilled in the art. Summary of the invention
[0010] One object of the present invention is to propose a method and device for optimizing sound pickup performance. The present invention adopts a dynamic convolution kernel selection mechanism, an improved hybrid neural network structure and a cross-channel self-attention mechanism to achieve high-quality enhancement of audio signals in complex environments. By combining generative adversarial networks with domain adaptation technology, the present invention can adaptively process diverse audio scenes and provide natural and clear audio output. The method has the advantages of strong adaptability, significant noise suppression effect, and high signal clarity. It significantly improves the quality of audio processing in complex environments and is widely applicable to multimedia and speech recognition and other fields.
[0011] A method for optimizing sound pickup performance according to an embodiment of the present invention comprises the following steps:
[0012] S1. Collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a dataset;
[0013] S2, preprocessing the data set, including noise reduction, normalization, and removal of invalid data;
[0014] S3, performing short-time Fourier transform on the preprocessed data set to convert the time domain signal into a frequency domain signal to obtain the frequency domain features of the audio signal;
[0015] S4. Use convolutional neural network to process frequency domain features, use dynamic convolution kernel selection mechanism to generate adaptive convolution kernel weights according to input frequency domain features, and extract fine frequency domain features;
[0016] S5, input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation;
[0017] S6. Use a bidirectional recurrent neural network to process the time series feature representation, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features;
[0018] S7, input the fused context features into the generator of the generative adversarial network, and optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, while introducing the domain adaptation technology to generate enhanced frequency domain features;
[0019] S8, performing inverse short-time Fourier transform on the enhanced frequency domain features and converting them back to time domain signals;
[0020] S9. Calculate the signal-to-noise ratio gain to evaluate the noise suppression effect, and output the final audio signal.
[0021] Furthermore, the S3 specifically includes:
[0022] S31, segment the preprocessed data set, each segment has a length of L samples, and add a window function to each segment of the signal, and the window function is a Hamming window:
[0023]
[0024] Where n represents the sample index in the window, and N represents the length of the window;
[0025] S32, performing fast Fourier transform on the windowed signal segment, converting the time domain signal into a frequency domain signal, and calculating the spectrum of each signal segment:
[0026]
[0027] Where X[k] represents the frequency domain signal, x[n] represents the windowed time domain signal, k represents the frequency index, and j represents the imaginary unit, which is the imaginary part in complex number operations.
[0028] S33, calculating the amplitude spectrum and phase spectrum of each frequency spectrum, the calculation formula of the amplitude spectrum is:
[0029]
[0030] Where |X[k]| represents the amplitude spectrum, the strength of the signal at frequency k, represents the real part of the frequency domain signal, Represents the imaginary part of the frequency domain signal;
[0031] The calculation formula of phase spectrum is:
[0032]
[0033] Where θ[k] represents the phase spectrum, the phase information at frequency k;
[0034] S34, using the amplitude spectrum information to analyze the frequency components of the audio signal, identifying and retaining the main frequency components, while suppressing unnecessary noise frequencies; using the phase spectrum information to maintain the time domain characteristics of the signal to ensure the naturalness of the signal during reconstruction;
[0035] S35, using a logarithmic average filter to smooth the frequency domain characteristics of each signal segment:
[0036]
[0037] in, represents the frequency domain feature after smoothing, M represents the size of the smoothing window, and ∈ represents a constant to avoid zero values in logarithmic calculation;
[0038] S36. Connect the smoothed frequency domain features of all signal segments to form a complete frequency domain feature sequence.
[0039] Furthermore, the S4 specifically includes:
[0040] S41, performing convolution processing on the complete frequency domain feature sequence in S36;
[0041] S42. According to the input data of the frequency domain characteristics, the convolution kernel weight is dynamically adjusted to generate multiple convolution kernels of different sizes, and the weight of the convolution kernel is calculated:
[0042]
[0043] in, represents the weight of the lth convolution kernel in the pth row and qth column, Y pq represents the value of the input frequency domain feature in the pth row and qth column, W (l) represents the weight coefficient of the lth convolution kernel, b (l) represents the bias term of the lth convolution kernel, R represents the number of rows of the input frequency domain feature matrix, S represents the number of columns of the input frequency domain feature matrix, and Y rs Represents the value of the input frequency domain feature in the rth row and sth column, Represents the association matrix between the lth convolution kernel and the input frequency domain feature matrix in the rth row and sth column;
[0044] S43. Perform convolution operations on convolution kernels of different sizes to generate multi-scale feature maps:
[0045]
[0046] in, represents the output value of the feature map in the pth row and qth column under the action of the lth convolution kernel, a ′ Indicates that the convolution kernel is in the ath ′ The offset of the row, b ′ Indicates that the convolution kernel is in the bth ′ The offset of the row;
[0047] S44, fusing the multi-scale feature maps, specifically including forming a complete feature representation matrix by channel splicing of the feature maps generated by all convolution kernels, and activating the matrix to generate a fine frequency domain feature representation;
[0048] S45. Input the fused feature representation matrix into the fully connected layer for further feature mapping and dimensionality reduction operations, and output the final refined frequency domain features.
[0049] Furthermore, the S5 specifically includes:
[0050] S51, input the refined frequency domain features in S45 into the improved long short-term memory network unit:
[0051] i t =σ(W i x t +U i h t-1 +V i c t-1 +b i );
[0052] f t =σ(W f x t +U f h t-1 +V f c t-1 +b f );
[0053] o t =σ(W o x t +U o h t-1 +V o c t-1 +b o );
[0054]
[0055] h t =o t ⊙tanh(c t );
[0056] Among them, i t represents the input gate, f t represents the forget gate, o t represents the output gate, represents the candidate memory cell state, c t represents the state of the memory unit, h t represents the hidden state, σ represents the activation function, tanh represents the hyperbolic tangent function, x t represents the input features of the current time step, h t-1 represents the hidden state of the previous time step, W i Represents the weight matrix of the current input of the input gate, U i The weight matrix representing the hidden state of the input gate one time step ago, V i The weight matrix representing the memory state of the input gate at the previous time step, b i Represents the bias value of the input gate, W f Represents the weight matrix of the current input of the forget gate, U f The weight matrix representing the hidden state of the forget gate one time step ago, V f The weight matrix representing the memory state of the forget gate at the previous time step, bf Represents the bias value of the forget gate, W o Represents the weight matrix of the current input of the output gate, U o The weight matrix representing the hidden state of the output gate one time step ago, V o The weight matrix representing the memory state of the output gate at the previous time step, b o Represents the bias value of the output gate, W c Represents the weight matrix of the current input of the candidate memory cell state, U c The weight matrix representing the hidden state of the candidate memory cell state in the previous time step, V c The weight matrix representing the memory state of the candidate memory unit state at the previous time step, b c represents the bias value of the candidate memory unit state, and γ represents the introduced adjustment coefficient, which is used to control the nonlinear interaction between the candidate memory state and the previous memory state;
[0057] in the formula Item passes through the previous memory state c t-1 With the current candidate memory state The dot product operation between strengthens the memory effect of historical information and weights new information. This nonlinear interaction can enhance the ability to capture long-term dependencies, allowing the model to better retain useful historical information when processing long-sequence data. Increasing the network's sensitivity to historical information helps maintain a more stable memory effect in long-sequence data, thereby improving the model's performance on long-term dependent information.
[0058] The calculation formulas of the input gate, forget gate, and output gate of the traditional LSTM only consider the current input and the hidden state of the previous time step. The improved LSTM additionally introduces the memory state c of the previous time step in the calculation of these gates. t-1 This makes the output of each gate not only depend on the current input and the previous hidden state, but also considers the impact of the previous memory state on the current gating. The complexity of the gating mechanism is enhanced, so that each gate can control the flow of information more flexibly, further refine the selectivity of important information, and improve the expressiveness of the model.
[0059] S52, the output hidden state h of the improved long short-term memory network unit t and the memory cell state c t Input to the improved gated recurrent unit:
[0060] z t =σ(W z x t +U z h t-1 +b z );
[0061] r t =σ(W r x t +U r h t-1 +b r );
[0062]
[0063] Among them, z t represents the update gate, r t Represents the reset gate, represents the candidate hidden state, h t represents the final hidden state, W z Represents the weight matrix of the current input of the update gate, U z represents the weight matrix of the hidden state of the previous time step of the update gate, W r Represents the weight matrix of the current input of the reset gate, U r Represents the weight matrix of the hidden state of the time step before resetting the gate, W h Represents the weight matrix of the current input of the candidate hidden state, U h The weight matrix representing the candidate hidden state, b z represents the bias value of the update gate, b r Represents the bias value of the reset gate, b h The bias value representing the candidate hidden state;
[0064] S53, through the hybrid structure of improved long short-term memory network and gated recurrent unit, the long-term and short-term dependencies in the audio signal are captured, and the final hidden state h t The output is represented as a time series feature.
[0065] Furthermore, the S6 specifically includes:
[0066] S61, inputting the temporal feature representation output by S53 into a bidirectional recurrent neural network, where the bidirectional recurrent neural network consists of a forward recurrent neural network and a backward recurrent neural network, and is used to capture bidirectional context information of the signal;
[0067] S62, the forward recurrent neural network processes the forward information of the input sequence, and the backward recurrent neural network processes the reverse information of the input sequence;
[0068] S63, concatenating the output hidden states of the forward recurrent neural network and the backward recurrent neural network to generate a bidirectional context feature representation;
[0069] S64. Introduce a cross-channel self-attention mechanism to weight the features of different channels by calculating the correlation between channels:
[0070]
[0071] Among them, A ij represents the attention weight of the i-th channel and the j-th channel, S ij represents the relevance score, Q i represents the feature representation of the i-th channel, K j represents the feature representation of the jth channel, and C represents the number of channels;
[0072] S65. Add the attention weight A ij Perform weighted calculation with channel features to obtain enhanced channel feature representation and output fused context features.
[0073] Furthermore, the S7 specifically includes:
[0074] S71, inputting the contextual features fused in S65 into the generator of the generative adversarial network to generate frequency domain features;
[0075] S72, performing denoising processing on the generated frequency domain features, the denoising processing including frequency domain filtering and noise suppression algorithm;
[0076] S73. At the same time as the generator outputs, a time-frequency domain consistency loss function is introduced. The loss function combines the consistency of the time domain and the frequency domain and takes into account the phase information:
[0077]
[0078] Among them, L consistency represents the consistency loss in the time-frequency domain, Represents the value of the real frequency domain feature at the u, v position, Indicates the value of generating frequency domain features at positions u and v. represents the phase angle of the kth frequency component of the real frequency domain feature at position u, v, λ represents the phase consistency weight parameter, U represents the number of rows of the frequency domain feature, V represents the number of columns of the frequency domain feature, and K represents the number of frequency components;
[0079] S74, the discriminator receives the generated features and the real features as input and calculates the authenticity score;
[0080] S75. Optimize the generator and discriminator through adversarial training;
[0081] S76. Combined with domain adaptation technology, the output features of the generator are dynamically adjusted to adapt to the changes in frequency domain features in different recording environments, ensuring the robustness of the generated frequency domain features in various environments and generating enhanced frequency domain features.
[0082] Furthermore, the S9 specifically includes:
[0083] S91, performing quality assessment on the time domain signal converted back in S8, specifically including calculating the signal-to-noise ratio of the signal:
[0084]
[0085] Among them, S i Represents the original audio signal value of the i-th sample point, N i represents the noise signal value of the i-th sample point, and N1 represents the total number of samples;
[0086] S92. Analyze the noise suppression effect of the enhanced audio signal, including calculating the total harmonic distortion of the signal before and after the enhancement:
[0087]
[0088] Among them, H k represents the amplitude of the audio signal at the kth harmonic frequency, and H1 represents the fundamental frequency amplitude of the audio signal;
[0089] S93. Further evaluate the audio signal quality by spectrum analysis method and calculate the spectrum distortion:
[0090]
[0091] Among them, X m It represents the amplitude of the original signal at the mth frequency point. represents the amplitude of the enhanced signal at the mth frequency point, and M1 represents the total number of frequency points;
[0092] S94. Comprehensively evaluate the final audio signal according to the calculated signal-to-noise ratio gain, total harmonic distortion, and spectrum distortion indicators, and output a final audio signal quality report according to these indicators.
[0093] A sound pickup performance optimization device according to an embodiment of the present invention includes:
[0094] The data acquisition module is used to collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a data set;
[0095] Data preprocessing module, used to preprocess the data set, including noise reduction, normalization, and removal of invalid data;
[0096] The frequency domain conversion module is used to perform short-time Fourier transform on the preprocessed data set, convert the time domain signal into a frequency domain signal, and obtain the frequency domain characteristics of the audio signal;
[0097] The convolution processing module is used to process the frequency domain features using a convolutional neural network, generate adaptive convolution kernel weights based on the input frequency domain features using a dynamic convolution kernel selection mechanism, and extract fine frequency domain features;
[0098] A temporal feature extraction module is used to input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation;
[0099] The context feature processing module is used to process the time series feature representation using a bidirectional recurrent neural network, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features;
[0100] The enhanced signal generation module is used to input the fused context features into the generator of the generative adversarial network, optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, and introduce the domain adaptation technology to generate enhanced frequency domain features;
[0101] A signal reconstruction module is used to perform an inverse short-time Fourier transform on the enhanced frequency domain features and convert them back to time domain signals;
[0102] The quality evaluation module is used to calculate the signal-to-noise ratio gain to evaluate the noise suppression effect and output the final audio signal.
[0103] The beneficial effects of the present invention are:
[0104] First of all, the present invention combines a dynamic convolution kernel selection mechanism, an improved hybrid neural network structure, a bidirectional recurrent neural network, a cross-channel self-attention mechanism, a generative adversarial network, and domain adaptation technology to significantly improve the quality and efficiency of audio signal processing. First, the dynamic convolution kernel selection mechanism enables the convolutional neural network to adaptively adjust the size and weight of the convolution kernel according to the frequency domain characteristics of the input, which not only improves the model's sensitivity to audio signals of different frequencies, but also enhances its adaptability in a diverse noise environment. Secondly, the improved hybrid neural network structure, by combining the long short-term memory network (LSTM) and the gated recurrent unit (GRU), effectively captures the long-term and short-term dependencies in the audio signal, reduces the risk of information loss, and ensures stability and accuracy in long sequence data processing.
[0105] In addition, the present invention further improves the ability to capture contextual information of audio signals by introducing a bidirectional recurrent neural network and a cross-channel self-attention mechanism. The bidirectional recurrent neural network can simultaneously process the forward and backward information of the signal, providing a richer context, while the cross-channel self-attention mechanism weights and enhances the features of important channels according to the correlation between the channels, thereby more accurately identifying and retaining important audio components and suppressing irrelevant noise.
[0106] Finally, in terms of audio signal generation and enhancement, the present invention adopts a combination of generative adversarial networks (GANs) and denoising strategies. By introducing a time-frequency domain consistency loss function, the consistency of the generated signal in the time domain and frequency domain is ensured, and signal distortion is reduced. The application of domain adaptation technology enables the model to dynamically adjust the output features according to different recording environments, thereby ensuring the quality of audio signals in various environments. This multi-level, multi-angle optimization design enables the present invention to output audio signals with high naturalness and clarity, effectively improving the noise suppression effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0107] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0108] Figure 1 A flow chart of a method and device for optimizing sound pickup performance proposed by the present invention;
[0109] Figure 2 The present invention provides a structural schematic diagram of a method and device for optimizing sound pickup performance. DETAILED DESCRIPTION
[0110] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.
[0111] refer to Figure 1 and Figure 2 , a method for optimizing sound pickup performance, comprising the following steps:
[0112] S1. Collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a dataset;
[0113] S2, preprocessing the data set, including noise reduction, normalization, and removal of invalid data;
[0114] S3, performing short-time Fourier transform on the preprocessed data set to convert the time domain signal into a frequency domain signal to obtain the frequency domain features of the audio signal;
[0115] S4. Use convolutional neural network to process frequency domain features, use dynamic convolution kernel selection mechanism to generate adaptive convolution kernel weights according to input frequency domain features, and extract fine frequency domain features;
[0116] S5, input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation;
[0117] S6. Use a bidirectional recurrent neural network to process the time series feature representation, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features;
[0118] S7, input the fused context features into the generator of the generative adversarial network, and optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, while introducing the domain adaptation technology to generate enhanced frequency domain features;
[0119] S8, performing inverse short-time Fourier transform on the enhanced frequency domain features and converting them back to time domain signals;
[0120] S9. Calculate the signal-to-noise ratio gain to evaluate the noise suppression effect, and output the final audio signal.
[0121] In this implementation, S3 specifically includes:
[0122] S31, segment the preprocessed data set, each segment has a length of L samples, and add a window function to each segment of the signal, and the window function is a Hamming window:
[0123]
[0124] Where n represents the sample index in the window, and N represents the length of the window;
[0125] S32, performing fast Fourier transform on the windowed signal segment, converting the time domain signal into a frequency domain signal, and calculating the spectrum of each signal segment:
[0126]
[0127] Where X[k] represents the frequency domain signal, x[n] represents the windowed time domain signal, k represents the frequency index, and j represents the imaginary unit, which is the imaginary part in complex number operations.
[0128] S33, calculating the amplitude spectrum and phase spectrum of each frequency spectrum, the calculation formula of the amplitude spectrum is:
[0129]
[0130] Where |X[k]| represents the amplitude spectrum, the strength of the signal at frequency k, represents the real part of the frequency domain signal, Represents the imaginary part of the frequency domain signal;
[0131] The calculation formula of phase spectrum is:
[0132]
[0133] Where θ[k] represents the phase spectrum, the phase information at frequency k;
[0134] S34, using the amplitude spectrum information to analyze the frequency components of the audio signal, identifying and retaining the main frequency components, while suppressing unnecessary noise frequencies; using the phase spectrum information to maintain the time domain characteristics of the signal to ensure the naturalness of the signal during reconstruction;
[0135] S35, using a logarithmic average filter to smooth the frequency domain characteristics of each signal segment:
[0136]
[0137] in, represents the frequency domain feature after smoothing, M represents the size of the smoothing window, and ∈ represents a constant to avoid zero values in logarithmic calculation;
[0138] S36. Connect the smoothed frequency domain features of all signal segments to form a complete frequency domain feature sequence.
[0139] In this implementation manner, the S4 specifically includes:
[0140] S41, performing convolution processing on the complete frequency domain feature sequence in S36;
[0141] S42. According to the input data of the frequency domain characteristics, the convolution kernel weight is dynamically adjusted to generate multiple convolution kernels of different sizes, and the weight of the convolution kernel is calculated:
[0142]
[0143] in, represents the weight of the lth convolution kernel in the pth row and qth column, Y pq represents the value of the input frequency domain feature in the pth row and qth column, W (l) represents the weight coefficient of the lth convolution kernel, b (l) represents the bias term of the lth convolution kernel, R represents the number of rows of the input frequency domain feature matrix, S represents the number of columns of the input frequency domain feature matrix, and Y rs Represents the value of the input frequency domain feature in the rth row and sth column, Represents the association matrix between the lth convolution kernel and the input frequency domain feature matrix in the rth row and sth column;
[0144] S43. Perform convolution operations on convolution kernels of different sizes to generate multi-scale feature maps:
[0145]
[0146] in, represents the output value of the feature map in the pth row and qth column under the action of the lth convolution kernel, a ′ Indicates that the convolution kernel is in the ath ′ The offset of the row, b ′ Indicates that the convolution kernel is in the bth ′ The offset of the row;
[0147] S44, fusing the multi-scale feature maps, specifically including forming a complete feature representation matrix by channel splicing of the feature maps generated by all convolution kernels, and activating the matrix to generate a fine frequency domain feature representation;
[0148] S45. Input the fused feature representation matrix into the fully connected layer for further feature mapping and dimensionality reduction operations, and output the final refined frequency domain features.
[0149] In this implementation manner, S5 specifically includes:
[0150] S51, input the refined frequency domain features in S45 into the improved long short-term memory network unit:
[0151] i t =σ(W i x t +U i h t-1 +V i c t-1 +b i );
[0152] f t =σ(W f x t +U f h t-1 +V f c t-1 +b f );
[0153] o t =σ(W o x t +U o h t-1 +V o c t-1 +b o );
[0154]
[0155]
[0156] h t =o t ⊙tanh(c t );
[0157] Among them, i t represents the input gate, f t represents the forget gate, o t represents the output gate, represents the candidate memory cell state, c t represents the state of the memory unit, h t represents the hidden state, σ represents the activation function, tanh represents the hyperbolic tangent function, x t represents the input features of the current time step, h t-1 represents the hidden state of the previous time step, W i Represents the weight matrix of the current input of the input gate, U i The weight matrix representing the hidden state of the input gate one time step ago, V i The weight matrix representing the memory state of the input gate at the previous time step, b i Represents the bias value of the input gate, W f Represents the weight matrix of the current input of the forget gate, U f The weight matrix representing the hidden state of the forget gate one time step ago, V f The weight matrix representing the memory state of the forget gate at the previous time step, b f Represents the bias value of the forget gate, W o Represents the weight matrix of the current input of the output gate, U o The weight matrix representing the hidden state of the output gate one time step ago, V o The weight matrix representing the memory state of the output gate at the previous time step, b o Represents the bias value of the output gate, W c Represents the weight matrix of the current input of the candidate memory cell state, U c The weight matrix representing the hidden state of the candidate memory cell state in the previous time step, V c The weight matrix representing the memory state of the candidate memory unit state at the previous time step, b c represents the bias value of the candidate memory unit state, and γ represents the introduced adjustment coefficient, which is used to control the nonlinear interaction between the candidate memory state and the previous memory state;
[0158] in the formula Item passes through the previous memory state c t-1With the current candidate memory state The dot product operation between strengthens the memory effect of historical information and weights new information. This nonlinear interaction can enhance the ability to capture long-term dependencies, allowing the model to better retain useful historical information when processing long-sequence data. Increasing the network's sensitivity to historical information helps maintain a more stable memory effect in long-sequence data, thereby improving the model's performance on long-term dependent information.
[0159] The calculation formulas of the input gate, forget gate, and output gate of the traditional LSTM only consider the current input and the hidden state of the previous time step. The improved LSTM additionally introduces the memory state c of the previous time step in the calculation of these gates. t-1 This makes the output of each gate not only depend on the current input and the previous hidden state, but also considers the impact of the previous memory state on the current gating. The complexity of the gating mechanism is enhanced, so that each gate can control the flow of information more flexibly, further refine the selectivity of important information, and improve the expressiveness of the model.
[0160] S52, the output hidden state h of the improved long short-term memory network unit t and the memory cell state c t Input to the improved gated recurrent unit:
[0161] z t =σ(W z x t +U z h t-1 +b z );
[0162] r t =σ(W r x t +U r h t-1 +b r );
[0163]
[0164] Among them, z t represents the update gate, r t Represents the reset gate, represents the candidate hidden state, h t represents the final hidden state, W z Represents the weight matrix of the current input of the update gate, U z represents the weight matrix of the hidden state of the previous time step of the update gate, W r Represents the weight matrix of the current input of the reset gate, U rRepresents the weight matrix of the hidden state of the time step before resetting the gate, W h Represents the weight matrix of the current input of the candidate hidden state, U h The weight matrix representing the candidate hidden state, b z represents the bias value of the update gate, b r Represents the bias value of the reset gate, b h The bias value representing the candidate hidden state;
[0165] S53, through the hybrid structure of improved long short-term memory network and gated recurrent unit, the long-term and short-term dependencies in the audio signal are captured, and the final hidden state h t The output is represented as a time series feature.
[0166] In this implementation manner, S6 specifically includes:
[0167] S61, inputting the temporal feature representation output by S53 into a bidirectional recurrent neural network, where the bidirectional recurrent neural network consists of a forward recurrent neural network and a backward recurrent neural network, and is used to capture bidirectional context information of the signal;
[0168] S62, the forward recurrent neural network processes the forward information of the input sequence, and the backward recurrent neural network processes the reverse information of the input sequence;
[0169] S63, concatenating the output hidden states of the forward recurrent neural network and the backward recurrent neural network to generate a bidirectional context feature representation;
[0170] S64. Introduce a cross-channel self-attention mechanism to weight the features of different channels by calculating the correlation between channels:
[0171]
[0172] Among them, A ij represents the attention weight of the i-th channel and the j-th channel, S ij represents the relevance score, Q i represents the feature representation of the i-th channel, K j represents the feature representation of the jth channel, and C represents the number of channels;
[0173] S65. Add the attention weight A ij Perform weighted calculation with channel features to obtain enhanced channel feature representation and output fused context features.
[0174] In this implementation manner, the S7 specifically includes:
[0175] S71, inputting the contextual features fused in S65 into the generator of the generative adversarial network to generate frequency domain features;
[0176] S72, performing denoising processing on the generated frequency domain features, the denoising processing including frequency domain filtering and noise suppression algorithm;
[0177] S73. At the same time as the generator outputs, a time-frequency domain consistency loss function is introduced. The loss function combines the consistency of the time domain and the frequency domain and takes into account the phase information:
[0178]
[0179] Among them, L consistency represents the consistency loss in the time-frequency domain, Represents the value of the real frequency domain feature at the u, v position, Indicates the value of generating frequency domain features at positions u and v. represents the phase angle of the kth frequency component of the real frequency domain feature at position u, v, λ represents the phase consistency weight parameter, U represents the number of rows of the frequency domain feature, V represents the number of columns of the frequency domain feature, and K represents the number of frequency components;
[0180] S74, the discriminator receives the generated features and the real features as input and calculates the authenticity score;
[0181] S75. Optimize the generator and discriminator through adversarial training;
[0182] S76. Combined with domain adaptation technology, the output features of the generator are dynamically adjusted to adapt to the changes in frequency domain features in different recording environments, ensuring the robustness of the generated frequency domain features in various environments and generating enhanced frequency domain features.
[0183] In this implementation manner, the S9 specifically includes:
[0184] S91, performing quality assessment on the time domain signal converted back in S8, specifically including calculating the signal-to-noise ratio of the signal:
[0185]
[0186] Among them, S i Represents the original audio signal value of the i-th sample point, N i represents the noise signal value of the i-th sample point, and N1 represents the total number of samples;
[0187] S92. Analyze the noise suppression effect of the enhanced audio signal, including calculating the total harmonic distortion of the signal before and after the enhancement:
[0188]
[0189] Among them, H k represents the amplitude of the audio signal at the kth harmonic frequency, and H1 represents the fundamental frequency amplitude of the audio signal;
[0190] S93. Further evaluate the audio signal quality by spectrum analysis method and calculate the spectrum distortion:
[0191]
[0192] Among them, X m It represents the amplitude of the original signal at the mth frequency point. represents the amplitude of the enhanced signal at the mth frequency point, and M1 represents the total number of frequency points;
[0193] S94. Comprehensively evaluate the final audio signal according to the calculated signal-to-noise ratio gain, total harmonic distortion, and spectrum distortion indicators, and output a final audio signal quality report according to these indicators.
[0194] A sound pickup performance optimization device, comprising:
[0195] The data acquisition module is used to collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a data set;
[0196] Data preprocessing module, used to preprocess the data set, including noise reduction, normalization, and removal of invalid data;
[0197] The frequency domain conversion module is used to perform short-time Fourier transform on the preprocessed data set, convert the time domain signal into a frequency domain signal, and obtain the frequency domain characteristics of the audio signal;
[0198] The convolution processing module is used to process the frequency domain features using a convolutional neural network, generate adaptive convolution kernel weights based on the input frequency domain features using a dynamic convolution kernel selection mechanism, and extract fine frequency domain features;
[0199] A temporal feature extraction module is used to input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation;
[0200] The context feature processing module is used to process the time series feature representation using a bidirectional recurrent neural network, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features;
[0201] The enhanced signal generation module is used to input the fused context features into the generator of the generative adversarial network, optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, and introduce the domain adaptation technology to generate enhanced frequency domain features;
[0202] A signal reconstruction module is used to perform an inverse short-time Fourier transform on the enhanced frequency domain features and convert them back to time domain signals;
[0203] The quality evaluation module is used to calculate the signal-to-noise ratio gain to evaluate the noise suppression effect and output the final audio signal.
[0204] Embodiment 1:
[0205] In order to verify the feasibility of the present invention in implementation, the experimental site of the present invention is set up in a large smart office building. This floor contains multiple conference rooms, open office areas and rest areas. The experimental time is May 2024, and the specific scenes include a large open office area and a small conference room. There are about 50 employees working at the same time in the open office area, and the environmental noise is complex, including keyboard tapping, chatting, telephone ringing, etc. In the small conference room, the experimental scene simulates an 8-person meeting, and the discussion involves technical details, requiring high audio clarity. The experimental equipment includes multiple smart speakers and conference recording systems, all equipped with the sound pickup performance optimization device of the present invention.
[0206] In the open office area, smart speakers are installed on desks to receive and process employees' voice commands. Due to the complex noise in the office area, traditional audio processing methods are difficult to effectively distinguish voice signals from background noise, resulting in low recognition rate of voice assistants and even inability to accurately execute commands. The device of the present invention collects real-time audio signals in the office area through a data acquisition module, and then performs data preprocessing, including noise reduction and normalization processing, to eliminate the interference of environmental noise. After short-time Fourier transform, the audio signal is converted into a frequency domain signal to obtain more refined frequency domain features. These features are processed by convolutional neural networks and dynamic convolution kernel selection mechanisms to extract clearer voice features. Next, the temporal feature extraction module and the bidirectional recurrent neural network process the context information, and combine the cross-channel self-attention mechanism to further optimize the voice signals of important channels. Finally, the enhanced frequency domain features are generated by the generator of the generative adversarial network, and the domain adaptation technology is used to adapt to changes in different environments. After signal reconstruction and quality evaluation, the smart speaker can understand user commands more accurately, and the recognition rate is significantly improved even in noisy environments.
[0207] In a small conference room, the meeting recording system uses the same audio processing flow to record and transcribe the meeting content in real time. The system uses a multi-channel sound pickup method, combined with context feature processing and denoising strategies to clearly separate the voice of each speaker. Generative adversarial networks further optimize the voice signal to ensure that the recorded text is accurate. In the experiment, through spectrum analysis and signal-to-noise ratio gain calculation, it was confirmed that the system can effectively suppress background noise, such as the sound of air conditioning and the sound of paper shuffling, while retaining important speech content.
[0208] In the test in the open office area, we selected 100 employees' voice commands for testing, including simple commands and complex questions.
[0209] Table 1 Comparison of key data between the method of the present invention and the traditional method in different scenarios
[0210]
[0211] In the open office area, the smart speaker faces complex background noise. The speech recognition rate of the traditional method is only 72%, while the speech recognition rate of the method of the present invention is as high as 93%, an increase of 21 percentage points. During lunch time with high background noise, the recognition rate of the traditional method further dropped to 58%, while the method of the present invention remained at 85%, showing a significant increase of 27 percentage points. In addition, after using the method of the present invention, the background noise suppression rate increased from 40% to 85%, an improvement of 45%. The audio processing time was also reduced from 1.5 seconds / time of the traditional method to 0.8 seconds / time, and the response speed was significantly accelerated. Based on the user questionnaire, user satisfaction increased from 65% to 90%, an increase of 25 percentage points. These data show that the method of the present invention greatly improves the recognition accuracy and response speed of the smart speaker in a high-noise environment, and significantly improves the user experience.
[0212] In a small conference room, the accuracy of the conference recording system was 80% when using the traditional method, but it was increased to 95% after using the method of the present invention, an increase of 15 percentage points. The signal-to-noise ratio gain increased from 8dB to 18dB, an increase of 10dB, which means that the clarity of speech has been significantly improved. The traditional method is generally effective in filtering out low-frequency noise, while the method of the present invention can significantly filter out low-frequency noise such as air-conditioning noise and paper shuffling noise. In terms of retaining high-frequency speech features, the traditional method has losses, while the method of the present invention can fully retain them without obvious distortion. As for the speech separation effect, the traditional method performs generally, while the method of the present invention can clearly separate the speech signals of different speakers, significantly improving the clarity of the speaker's speech. In addition, the audio processing time is reduced from 2 seconds / segment to 1 second / segment, which improves the recording efficiency.
[0213] In summary, the method of the present invention is significantly superior to the traditional method in different scenarios. In the open office area, the recognition rate and response speed of the smart speaker in a complex noise environment are greatly improved, and user satisfaction is significantly improved. In the small conference room, the accuracy, signal-to-noise ratio gain, low-frequency noise filtering effect, high-frequency speech feature retention and speech separation effect of the conference recording system are significantly improved, the processing time is shortened, and the recording efficiency is improved. These data fully demonstrate the superiority and practicality of the method of the present invention in a complex audio environment.
[0214] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.
Claims
1. A method for optimizing sound pickup performance, characterized in that: The steps include: S1. Collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a dataset; S2, preprocessing the data set, including noise reduction, normalization, and removal of invalid data; S3, performing short-time Fourier transform on the preprocessed data set to convert the time domain signal into a frequency domain signal to obtain the frequency domain features of the audio signal; S4. Use convolutional neural network to process frequency domain features, use dynamic convolution kernel selection mechanism to generate adaptive convolution kernel weights according to input frequency domain features, and extract fine frequency domain features; S5, input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation; S6. Use a bidirectional recurrent neural network to process the time series feature representation, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features; S7, input the fused context features into the generator of the generative adversarial network, and optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, while introducing the domain adaptation technology to generate enhanced frequency domain features; S8, performing inverse short-time Fourier transform on the enhanced frequency domain features and converting them back to time domain signals; S9. Calculate the signal-to-noise ratio gain to evaluate the noise suppression effect, and output the final audio signal.
2. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S3 specifically includes: S31, segment the preprocessed data set, each segment has a length of L samples, and add a window function to each segment of the signal, and the window function is a Hamming window: Where n represents the sample index in the window, and N represents the length of the window; S32, performing fast Fourier transform on the windowed signal segment, converting the time domain signal into a frequency domain signal, and calculating the spectrum of each signal segment: Where X[k] represents the frequency domain signal, x[n] represents the windowed time domain signal, k represents the frequency index, and j represents the imaginary unit, which is the imaginary part in complex number operations. S33, calculating the amplitude spectrum and phase spectrum of each frequency spectrum, the calculation formula of the amplitude spectrum is: Where |X[k]| represents the amplitude spectrum, the strength of the signal at frequency k, represents the real part of the frequency domain signal, Represents the imaginary part of the frequency domain signal; The calculation formula of phase spectrum is: Where θ[k] represents the phase spectrum, the phase information at frequency k; S34, using the amplitude spectrum information to analyze the frequency components of the audio signal, identifying and retaining the main frequency components, while suppressing unnecessary noise frequencies; using the phase spectrum information to maintain the time domain characteristics of the signal; S35, using a logarithmic average filter to smooth the frequency domain characteristics of each signal segment: in, represents the frequency domain feature after smoothing, M represents the size of the smoothing window, and ∈ represents a constant to avoid zero values in logarithmic calculation; S36. Connect the smoothed frequency domain features of all signal segments to form a complete frequency domain feature sequence.
3. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S4 specifically includes: S41, performing convolution processing on the complete frequency domain feature sequence in S36; S42. According to the input data of the frequency domain characteristics, the convolution kernel weight is dynamically adjusted to generate multiple convolution kernels of different sizes, and the weight of the convolution kernel is calculated: in, represents the weight of the lth convolution kernel in the pth row and qth column, Y pq represents the value of the input frequency domain feature in the pth row and qth column, W (l) represents the weight coefficient of the lth convolution kernel, b (l) represents the bias term of the lth convolution kernel, R represents the number of rows of the input frequency domain feature matrix, S represents the number of columns of the input frequency domain feature matrix, and Y rs Represents the value of the input frequency domain feature in the rth row and sth column, Represents the association matrix between the lth convolution kernel and the input frequency domain feature matrix in the rth row and sth column; S43. Perform convolution operations on convolution kernels of different sizes to generate multi-scale feature maps: in, represents the output value of the feature map in the pth row and qth column under the action of the lth convolution kernel, a ′ Indicates that the convolution kernel is in the ath ′ The offset of the row, b ′ Indicates that the convolution kernel is in the bth ′ The offset of the row; S44, fusing the multi-scale feature maps, specifically including forming a complete feature representation matrix by channel splicing of the feature maps generated by all convolution kernels, and activating the matrix to generate a fine frequency domain feature representation; S45. Input the fused feature representation matrix into the fully connected layer for further feature mapping and dimensionality reduction operations, and output the final refined frequency domain features.
4. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S5 specifically includes: S51, input the refined frequency domain features in S45 into the improved long short-term memory network unit: i t =σ(W i x t +U i h t-1 +V i c t-1 +b i ); f t =σ(W f x t +U f h t-1 +V f c t-1 +b f ); o t =σ(W o x t +U o h t-1 +V o c t-1 +b o ); h t =o t ⊙tanh(c t ); Among them, i t represents the input gate, f t represents the forget gate, o t represents the output gate, represents the candidate memory cell state, c t represents the state of the memory unit, h t represents the hidden state, σ represents the activation function, tanh represents the hyperbolic tangent function, x t represents the input features of the current time step, h t-1 represents the hidden state of the previous time step, W i Represents the weight matrix of the current input of the input gate, U i The weight matrix representing the hidden state of the input gate one time step ago, V i The weight matrix representing the memory state of the input gate at the previous time step, b i Represents the bias value of the input gate, W f Represents the weight matrix of the current input of the forget gate, U f The weight matrix representing the hidden state of the forget gate one time step ago, V f The weight matrix representing the memory state of the forget gate at the previous time step, b f Represents the bias value of the forget gate, W o Represents the weight matrix of the current input of the output gate, U o The weight matrix representing the hidden state of the output gate one time step ago, V o The weight matrix representing the memory state of the output gate at the previous time step, b o Represents the bias value of the output gate, W c Represents the weight matrix of the current input of the candidate memory cell state, U c The weight matrix representing the hidden state of the candidate memory cell state in the previous time step, V c The weight matrix representing the memory state of the candidate memory unit state at the previous time step, b c represents the bias value of the candidate memory unit state, and γ represents the introduced adjustment coefficient, which is used to control the nonlinear interaction between the candidate memory state and the previous memory state; S52, the output hidden state h of the improved long short-term memory network unit t and the memory cell state c t Input to the improved gated recurrent unit: z t =σ(W z x t +U z h t-1 +b z ); r t =σ(W r x t +U r h t-1 +b r ); Among them, z t represents the update gate, r t Represents the reset gate, represents the candidate hidden state, h t represents the final hidden state, W z Represents the weight matrix of the current input of the update gate, U z represents the weight matrix of the hidden state of the previous time step of the update gate, W r Represents the weight matrix of the current input of the reset gate, U r Represents the weight matrix of the hidden state of the time step before resetting the gate, W h Represents the weight matrix of the current input of the candidate hidden state, U h The weight matrix representing the candidate hidden state, b z represents the bias value of the update gate, b r Represents the bias value of the reset gate, b h The bias value representing the candidate hidden state; S53, through the hybrid structure of improved long short-term memory network and gated recurrent unit, the long-term and short-term dependencies in the audio signal are captured, and the final hidden state h t The output is represented as a time series feature.
5. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S6 specifically includes: S61, inputting the temporal feature representation output by S53 into a bidirectional recurrent neural network, where the bidirectional recurrent neural network consists of a forward recurrent neural network and a backward recurrent neural network; S62, the forward recurrent neural network processes the forward information of the input sequence, and the backward recurrent neural network processes the reverse information of the input sequence; S63, concatenating the output hidden states of the forward recurrent neural network and the backward recurrent neural network to generate a bidirectional context feature representation; S64. Introduce a cross-channel self-attention mechanism to weight the features of different channels by calculating the correlation between channels: Among them, A ij represents the attention weight of the i-th channel and the j-th channel, S ij represents the relevance score, Q i represents the feature representation of the i-th channel, K j represents the feature representation of the jth channel, and C represents the number of channels; S65. Add the attention weight A ij Perform weighted calculation with channel features to obtain enhanced channel feature representation and output fused context features.
6. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S7 specifically includes: S71, inputting the contextual features fused in S65 into the generator of the generative adversarial network to generate frequency domain features; S72, performing denoising processing on the generated frequency domain features, the denoising processing including frequency domain filtering and noise suppression algorithm; S73. At the same time as the generator outputs, a time-frequency domain consistency loss function is introduced. The loss function combines the consistency of the time domain and the frequency domain and takes into account the phase information: Among them, L consistency represents the consistency loss in the time-frequency domain, Represents the value of the real frequency domain feature at the u, v position, Indicates the value of generating frequency domain features at positions u and v. represents the phase angle of the kth frequency component of the real frequency domain feature at position u, v, λ represents the phase consistency weight parameter, U represents the number of rows of the frequency domain feature, V represents the number of columns of the frequency domain feature, and K represents the number of frequency components; S74, the discriminator receives the generated features and the real features as input and calculates the authenticity score; S75. Optimize the generator and discriminator through adversarial training; S76. Combine domain adaptation technology to dynamically adjust the output features of the generator to generate enhanced frequency domain features.
7. A method for optimizing sound pickup performance according to claim 1, characterized in that: The S9 specifically includes: S91, performing quality assessment on the time domain signal converted back in S8, specifically including calculating the signal-to-noise ratio of the signal: Among them, S i Represents the original audio signal value of the i-th sample point, N i represents the noise signal value of the i-th sample point, and N1 represents the total number of samples; S92. Analyze the noise suppression effect of the enhanced audio signal, including calculating the total harmonic distortion of the signal before and after the enhancement: Among them, H k represents the amplitude of the audio signal at the kth harmonic frequency, and H1 represents the fundamental frequency amplitude of the audio signal; S93. Further evaluate the audio signal quality by spectrum analysis method and calculate the spectrum distortion: Among them, X m It represents the amplitude of the original signal at the mth frequency point. represents the amplitude of the enhanced signal at the mth frequency point, and M1 represents the total number of frequency points; S94. Comprehensively evaluate the final audio signal according to the calculated signal-to-noise ratio gain, total harmonic distortion, and spectrum distortion indicators, and output a final audio signal quality report according to these indicators.
8. A device for optimizing sound pickup performance, characterized in that: include: The data acquisition module is used to collect audio signals, including recording data in indoor, outdoor, quiet and noisy environments, and build a data set; Data preprocessing module, used to preprocess the data set, including noise reduction, normalization, and removal of invalid data; The frequency domain conversion module is used to perform short-time Fourier transform on the preprocessed data set, convert the time domain signal into a frequency domain signal, and obtain the frequency domain characteristics of the audio signal; The convolution processing module is used to process the frequency domain features using a convolutional neural network, generate adaptive convolution kernel weights based on the input frequency domain features using a dynamic convolution kernel selection mechanism, and extract fine frequency domain features; A temporal feature extraction module is used to input the refined frequency domain features into the hybrid structure of the improved long short-term memory network and the gated recurrent unit, and output the temporal feature representation; The context feature processing module is used to process the time series feature representation using a bidirectional recurrent neural network, obtain bidirectional context information, and combine the cross-channel self-attention mechanism to weighted enhance the features of important channels and output the fused context features; The enhanced signal generation module is used to input the fused context features into the generator of the generative adversarial network, optimize the generated features through the denoising strategy and the time-frequency domain consistency loss function, and introduce the domain adaptation technology to generate enhanced frequency domain features; A signal reconstruction module is used to perform an inverse short-time Fourier transform on the enhanced frequency domain features and convert them back to time domain signals; The quality evaluation module is used to calculate the signal-to-noise ratio gain to evaluate the noise suppression effect and output the final audio signal.
Citation Information
Cited By
Comprehensive online monitoring system for cable branch box
CN120074006A
Method and device for adjusting power frequency band of carrier communication system
CN120499832A
Big data-based intelligent analysis method for running track of electric bicycle
CN121786616A
