High-fidelity music noise reduction method and system based on complex environment

Abstract

The invention discloses a high-fidelity music noise reduction method and system based on a complex environment, and the method comprises the steps: constructing a noise reduction model which comprises a feature extraction module, a noise representation learning module and a noise filtering module; performing feature transformation, feature evaluation and frequency channel expansion on the audio segments containing noise, optimizing noise reduction model parameters, and performing early feature extraction on the expanded audio segments by using a feature extraction module to obtain original audio features containing noise; noise in the original noise-containing frequency segment is subjected to representation learning and reinforcement by using a noise representation learning module to obtain enhanced noise features; and based on the enhanced noise feature, removing and refining a noise part in the original noise-containing frequency feature by using a noise filtering module, and obtaining a denoised high-fidelity music clip after inverse feature transformation. By means of the method, high-fidelity music noise reduction in the complex environment can be completed, and the defects that data are scarce and complex scenes are difficult to construct are overcome.

Description

Technical Field

[0001] The present invention relates to the field of audio enhancement technology, and in particular to a high-fidelity music noise reduction method and system based on complex environments. Background Art

[0002] Since the advent of recording technology, the way audio is recorded and transmitted has undergone tremendous development, but noise pollution has always been a major interference that reduces sound quality. For historical recordings, early technical limitations led to sound degradation (such as background noise and popping sounds). For modern audio, the advent of mobile devices has spawned a large amount of music content recorded in non-professional environments (such as street performances and improvisations), and the high degree of mixing of environmental noise (such as wind and human voices) with the sound source significantly reduces the listening experience. Using music noise reduction technology to reduce interfering noise in audio and improve sound quality is not only a core requirement of audio restoration, but also an important means to meet the public's high-fidelity listening needs.

[0003] For example, Chinese patent document CN115083440A discloses a method for reducing noise in an audio signal, comprising: performing spectrum transformation on a music audio signal to obtain spectrum information of the music audio signal; determining a noise reduction reference frequency based on at least one music characteristic frequency band in the spectrum information; and performing noise reduction processing on the music audio signal based on the noise reduction reference frequency.

[0004] A Chinese patent document with publication number CN116524944A discloses an audio noise reduction method, comprising: obtaining an audio signal to be noise reduced, performing time-frequency conversion processing on the audio signal to be noise reduced to obtain a frequency domain signal to be noise reduced; extracting acoustic features of a preset dimension of the frequency domain signal to be noise reduced; determining a first probability that a music signal exists in the audio signal to be noise reduced based on the acoustic features; and performing noise reduction processing on the frequency domain signal to be noise reduced based on the first probability and an amplitude spectrum of the frequency domain signal to be noise reduced to obtain a noise-reduced audio signal corresponding to the audio signal to be noise reduced.

[0005] Despite decades of audio restoration research, traditional methods struggle to accurately separate frequency domain masks in complex noise scenarios due to the high frequency overlap between noise and music signals. Furthermore, non-stationary noise (e.g., sudden whistle sounds) is difficult to accurately model due to its time-varying nature. Furthermore, the current application of various deep learning models to music noise reduction has yet to achieve the goal of accurate noise reduction in complex environments. The main research difficulties lie in the scarcity of high-quality clean audio-noise frequency pairing data, the difficulty in completing model training, the difficulty in constructing complex noise scenarios, and the poor noise reduction performance of the models, all of which limit the effectiveness of high-fidelity music noise reduction in complex environments. Summary of the Invention

[0006] The present invention provides a high-fidelity music noise reduction method and system based on complex environments, which can achieve high-fidelity music noise reduction in complex environments and make up for the defects of data scarcity and difficulty in constructing complex scenes.

[0007] A high-fidelity music noise reduction method based on a complex environment includes the following steps:

[0008] (1) constructing a noise reduction model, wherein the noise reduction model includes a feature extraction module, a noise representation learning module, and a noise filtering module;

[0009] (2) Perform feature transformation, feature evaluation, and frequency channel expansion on the noisy audio segment, optimize the noise reduction model parameters, and use the feature extraction module to perform early feature extraction on the expanded audio segment to obtain the original noisy audio feature;

[0010] (3) Using the noise representation learning module to learn and enhance the noise in the original noisy frequency segment, the enhanced noise feature is obtained;

[0011] (4) Based on the enhanced noise features, the noise filtering module removes and refines the noise part of the original noisy frequency features, and obtains the high-fidelity music clip after noise reduction after inverse feature transformation.

[0012] In step (1), the noise representation learning module includes an encoder and a decoder, which adopts a mirror-symmetric design and a jump connection between the encoder and the decoder. At the same time, a three-dimensional attention mechanism is introduced between the encoder and the decoder to adaptively allocate attention weights, enhance the time and frequency regions dominated by noise, and suppress the low-energy noise regions overlapping with the music signal, thereby improving the noise representation modeling capability.

[0013] The specific process of step (2) is:

[0014] (2-1) Perform short-time Fourier transform on the noise-containing frequency segment to obtain a dual-channel image;

[0015] (2-2) Statistical feature evaluation is performed based on dual-channel images, and the noise reduction model parameters are dynamically optimized through feature combination;

[0016] (2-3) The channels of the dual-channel image data are expanded based on the frequency position embedding, and the feature extraction module is used to perform early feature extraction to obtain the original noisy frequency features.

[0017] Step (2-2) specifically includes:

[0018] Based on the dual-channel images, three statistical features of the noisy frequency segments were calculated, including energy variation, zero-crossing rate, and kurtosis. Energy variation is used to measure the energy fluctuation of the audio signal in the time domain and is quantified by calculating the standard deviation of the short-term energy in each frame. The zero-crossing rate is an indicator of the proportion of high-frequency components in the audio signal and is defined as the number of times the signal crosses the zero axis per unit time. Kurtosis reflects the degree of signal spikes and is the ratio of the fourth-order central moment to the square of the variance.

[0019] A feature combination-driven evaluation mechanism is used to determine whether burst noise occurs and dynamically optimize the noise reduction model parameters.

[0020] Steps (2-3) specifically include:

[0021] The dual-channel image is expanded by 10 additional channels using frequency position embedding, adding frequency-related features and expanding the data to 12 channels. The principles are as follows:

[0022]

[0023] Where FP is the frequency position embedding vector, F is the frequency bandwidth, and i is any integer from 1 to F. The expanded data is processed using a feature extractor containing a convolutional layer and an activation function to obtain the original noisy frequency features.

[0024] The specific process of step (3) is:

[0025] (3-1) Using the noise representation learning module based on the three-dimensional attention mechanism, the attention weight is adaptively assigned to learn and model the noise in the original noisy frequency features to obtain the noise features;

[0026] (3-2) Based on the supervised attention module, feature weighting is performed to enhance the noise feature and obtain the enhanced noise feature.

[0027] Step (3-2) specifically includes:

[0028] The supervised attention module is used to complete feature weighting, and the noise features are convolved to generate a residual image, which is added to the original noisy frequency features for further convolution and activation to generate an attention map. The noise features are adjusted according to the attention map to generate an enhanced noise feature that only contains the features most relevant to the noise.

[0029] The specific process of step (4) is:

[0030] Based on the enhanced noise features, the noise filtering module is used to remove the noise part of the original noisy audio features and complete the feature refinement to obtain the denoised audio features; then, based on the inverse short-time Fourier transform, the denoised audio features are restored to the denoised high-fidelity music clips.

[0031] A high-fidelity music noise reduction system based on a complex environment includes a memory and one or more processors. The memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned high-fidelity music noise reduction method.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention utilizes a noise representation learning module based on a three-dimensional attention mechanism and a supervised attention module to complete noise modeling and enhancement, and uses a noise filtering module to complete denoising and refinement, thereby achieving high-fidelity music noise reduction in complex scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of a high-fidelity music noise reduction method based on a complex environment according to an embodiment of the present invention.

[0035] Figure 2 1 is a flow chart of data enhancement and noisy frequency segment synthesis in one embodiment of the present invention;

[0036] Figure 3 1. A schematic diagram of a flow chart for early processing of a noisy audio segment according to an embodiment of the present invention;

[0037] Figure 4 The figure is a flow chart of characterizing, learning and enhancing noise features in one embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be described in further detail below with reference to the accompanying drawings and examples. It should be noted that the following examples are intended to facilitate understanding of the present invention and do not have any limiting effect on the present invention.

[0039] like Figure 1 As shown, a high-fidelity music noise reduction method based on a complex environment includes the following steps:

[0040] S101, using data augmentation methods to amplify pure noise audio clips, and combining them with clean audio clips to synthesize noisy music clips.

[0041] Based on the precise text description of the complex noise environment, a generative model is used to generate audio clips that meet the text description and natural noise standards to amplify the pure noise audio data. Based on the random time domain sampling strategy, pure noise clips of specified length are randomly intercepted and mixed with clean audio clips at a specified signal-to-noise ratio to achieve non-aligned superposition in the time dimension and obtain noisy music clips.

[0042] S102, performing feature transformation, feature evaluation and frequency channel expansion on the noisy music clip, and obtaining the original noisy frequency features after early feature extraction.

[0043] A short-time Fourier transform is performed on the noisy music clip to obtain a dual-channel image, and the statistical features of the dual-channel image are evaluated. Its energy change, zero-crossing rate and kurtosis are analyzed. The presence of sudden noise is determined by feature combination, and the model parameters are dynamically optimized. The channels of the dual-channel image data are expanded by frequency position embedding, and early feature extraction is performed on the expanded data to obtain the original noisy frequency features.

[0044] S103: Based on the original noisy audio frequency features, the noise in the audio is characterized and enhanced to obtain enhanced noise features.

[0045] A noise representation learning module based on the three-dimensional attention mechanism is used to adaptively allocate attention weights, enhance the noise-dominated time-frequency region, learn and model the noise part of the original noisy frequency-frequency features, and obtain noise features. The supervised attention module is then used to weight the noise features and strengthen the noise features to obtain enhanced noise features.

[0046] S104: Based on the enhanced noise features, the original noisy audio features are denoised and refined, and the denoised high-fidelity music clip is restored using an inverse feature transformation.

[0047] Based on the enhanced noise features, the noise filtering module is used to remove the noise part of the original noisy frequency features and complete the refinement of the audio features to obtain the denoised audio features. The inverse short-time Fourier transform is used to restore the audio features to the denoised high-fidelity music clips.

[0048] The above-mentioned high-fidelity music noise reduction method for complex environments first uses data augmentation methods to amplify pure noise audio clips, combines them with clean audio clips to synthesize noisy music clips, and then performs feature transformation, feature evaluation, and frequency channel expansion on the noisy music clips. Early feature extraction is performed based on the expanded audio data to obtain the original noisy audio features. The noise in the audio is then characterized and enhanced to obtain enhanced noise features. Based on the enhanced noise features, the noise portion of the original noisy audio features is removed and refined. Finally, after inverse feature transformation, the noise-reduced high-fidelity music clip is restored. The present invention uses data augmentation to supplement the expanded clean audio-noisy audio pairing data to guide model training, utilizes a noise representation learning module and a supervised attention module based on a three-dimensional attention mechanism to complete noise modeling and enhancement, and uses a noise filtering module to complete denoising and refinement, thereby achieving high-fidelity music noise reduction in complex scenarios.

[0049] Based on the above embodiment, Figure 2 The steps of data enhancement and synthesis of noisy frequency segments are decomposed and refined. Figure 2 As shown, the implementation process includes the following:

[0050] S201, based on the precise text description of the complex noise environment, a generative model is used to generate a segment-amplified pure noise audio data that meets the natural noise standard.

[0051] The audio generation model has the ability to generate acoustic objects driven by text and the flexibility of spectral modeling. Based on precise text clues, it contains an accurate description of complex noise environments, guiding the model to simulate the broadband characteristics of natural noise and generate pure noise audio, making up for the shortcomings of existing data. It adds audio that meets the standards of real and complex environmental noise to the training dataset, thereby improving the model's generalization performance.

[0052] S202, based on a random time domain sampling strategy, realize non-aligned superposition of pure noise audio segments and clean audio segments in the time dimension to synthesize noisy audio segments.

[0053] A random time-domain sampling strategy is used to randomly determine the starting point in a pure noise segment, intercept an audio segment of a specified length, and mix it with the clean audio segment at a specific signal-to-noise ratio. This achieves non-aligned superposition of audio samples in the time dimension, simulating the arbitrariness of noise sources in the real world and improving the limitations of the traditional fixed-start point interception method. The signal-to-noise ratio is calculated as follows:

[0054] SNR(R M (t))=10lg(P C / P N )

[0055] Among them, R M (t) is the noise frequency, P C and P N They represent the energy of the clean audio part and the pure noise audio part in the noisy frequency respectively. The specific method of synthesizing the noisy frequency is as follows:

[0056] R M (t) = R C (t)+αR N (t)

[0057] Among them, R C (t) is the clean audio, R N (t) is pure noise audio, α represents the signal-to-noise ratio scaling factor, and the value of α is set to a random value between [2dB, 20dB] to enhance the diversity of the dataset and improve the generalization ability of the model.

[0058] It can be understood that this embodiment provides a possible implementation method for using the data enhancement method to amplify pure noise audio clips and combine them with clean audio clips to synthesize noisy music clips, which lays the foundation for subsequent early processing of noisy audio clips.

[0059] Based on the above embodiment, Figure 3 The steps of early processing of noisy frequency segments are decomposed and refined. Figure 3 As shown, the implementation process includes the following:

[0060] S301 , performing short-time Fourier transform on the music clip containing noise to obtain a dual-channel image.

[0061] For input noise music R M (t) Perform short-time Fourier transform to output the real and imaginary parts, and combine the real and imaginary parts to obtain a dual-channel image F MRI .

[0062] S302, performing statistical feature evaluation based on the dual-channel image, and dynamically optimizing model parameters through feature combination.

[0063] Based on F MRI Calculated R M (t) is characterized by three statistical features: energy variation, zero-crossing rate, and kurtosis. Energy variation measures the energy fluctuation of the signal in the time domain and is quantified by calculating the standard deviation of the short-term energy in each frame. The zero-crossing rate reflects the proportion of high-frequency components in the signal and is defined as the number of times the signal crosses the zero axis per unit time. Kurtosis reflects the degree of signal spikes and is the ratio of the fourth-order central moment to the square of the variance. A feature combination-driven evaluation mechanism is used to determine whether burst noise has occurred and dynamically optimize the denoising model parameters.

[0064] S303 , embedding and expanding the channels of the dual-channel image data based on the frequency position, and performing early feature extraction to obtain original noisy frequency features.

[0065] In order to provide frequency information for frequency-aware processing, we use frequency position embedding to transform F MRI We expanded 10 additional channels, added frequency-related features, and expanded the data to 12 channels. The principles are as follows:

[0066]

[0067] Where FP is the frequency position embedding vector, F is the frequency bandwidth, and i can be any integer from 1 to F. The expanded data is processed using a feature extractor consisting of a convolutional layer and an activation function to obtain the original noisy frequency features.

[0068] It can be understood that this embodiment provides a possible implementation method for performing feature transformation, feature evaluation and frequency channel expansion on a noisy music clip, and performing early feature extraction on the expanded audio data to obtain the original noisy audio features, which lays the foundation for subsequent noise representation learning and enhancement.

[0069] Based on the above embodiment, Figure 4The steps of characterization learning and enhancement of noise features are decomposed and refined. Figure 4 As shown, the implementation process includes the following:

[0070] S401, using a noise representation learning module based on a three-dimensional attention mechanism, adaptively assigning attention weights, learning and modeling the noise in the original noisy frequency features, and obtaining noise features.

[0071] The noise representation learning module consists of an encoder and a decoder, which adopts a mirror-symmetric design. Skip connections are used between symmetric modules to mitigate gradient vanishing. To address the semantic confusion caused by the simple feature superposition in traditional skip connections, a three-dimensional attention mechanism is introduced between the encoder and decoder to adaptively allocate attention weights, enhance the time and frequency regions dominated by noise, and suppress the low-energy noise regions that overlap with the music signal, further improving the noise representation modeling capability. The principle is as follows: the channel of the feature map is C, the height is H, and the width is W. Each pixel of the feature map is regarded as a neuron, and weights are assigned to different spatial positions by calculating the energy function. The minimum energy of a neuron is calculated as follows:

[0072]

[0073] Where t represents the target neuron, and is the mean and variance of other neurons in the same channel; λ is the regularization coefficient, x i are other neurons. M = H × W is the number of neurons in a single channel. The significance of each neuron can be used The weights of each neuron are calculated as follows:

[0074]

[0075] in, and X are the input feature map and output feature map, and the sigmoid activation function is used to limit The above three-dimensional attention mechanism adjusts the weight distribution of each position and enhances the noise-dominated area to reduce semantic confusion, improve noise modeling capabilities, and obtain noise features.

[0076] S402: performing noise feature weighting and enhancing on the basis of the supervised attention module to obtain enhanced noise features.

[0077] The supervised attention module is used to complete feature weighting, and the noise features are convolved to generate a residual image, which is added to the original noisy frequency features for further convolution and activation to generate an attention map. The noise features are adjusted according to the attention map to generate an enhanced noise feature that only contains the features most relevant to the noise.

[0078] It can be understood that this embodiment provides a possible implementation method for characterizing, learning, and strengthening the noise in the audio based on the original noisy frequency features to obtain enhanced noise features, which lays the foundation for subsequent denoising, refinement, and restoration.

[0079] The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A high-fidelity music noise reduction method based on complex environments, characterized in that: The following steps are involved: (1) constructing a noise reduction model, wherein the noise reduction model includes a feature extraction module, a noise representation learning module, and a noise filtering module; (2) Perform feature transformation, feature evaluation, and frequency channel expansion on the noisy audio segment, optimize the noise reduction model parameters, and use the feature extraction module to perform early feature extraction on the expanded audio segment to obtain the original noisy audio feature; (3) Using the noise representation learning module to learn and enhance the noise in the original noisy frequency features, an enhanced noise feature is obtained; (4) Based on the enhanced noise feature, the noise filtering module is used to remove and refine the noise part of the original noisy frequency feature, and after inverse feature transformation, the high-fidelity music clip with noise reduction is obtained.

2. The high-fidelity music noise reduction method based on complex environment according to claim 1, characterized in that: In step (1), the noise representation learning module includes an encoder and a decoder, which adopts a mirror-symmetric design and a jump connection between the encoder and the decoder. At the same time, a three-dimensional attention mechanism is introduced between the encoder and the decoder to adaptively allocate attention weights, enhance the time and frequency regions dominated by noise, and suppress the low-energy noise regions overlapping with the music signal, thereby improving the noise representation modeling capability.

3. The high-fidelity music noise reduction method based on complex environment according to claim 1, characterized in that: The specific process of step (2) is: (2-1) Perform short-time Fourier transform on the noise-containing frequency segment to obtain a dual-channel image; (2-2) Statistical feature evaluation is performed based on dual-channel images, and the noise reduction model parameters are dynamically optimized through feature combination; (2-3) The channels of the dual-channel image data are expanded based on the frequency position embedding, and the feature extraction module is used to perform early feature extraction to obtain the original noisy frequency features.

4. The high-fidelity music noise reduction method based on complex environment according to claim 3, characterized in that: Step (2-2) specifically includes: Three statistical features of noisy frequency segments are calculated based on dual-channel images, including energy variation, zero-crossing rate, and kurtosis. Energy variation is used to measure the degree of energy fluctuation of the audio signal in the time domain and is quantified by calculating the standard deviation of the short-term energy in each frame. The zero-crossing rate is an indicator of the proportion of high-frequency components in the audio signal and is defined as the number of times the signal crosses the zero axis per unit time. Kurtosis reflects the degree of signal spikes and is the ratio of the fourth-order central moment to the square of the variance. A feature combination-driven evaluation mechanism is used to determine whether burst noise occurs and dynamically optimize the noise reduction model parameters.

5. The high-fidelity music noise reduction method based on complex environment according to claim 3, characterized in that: Steps (2-3) specifically include: The dual-channel image is expanded by 10 additional channels using frequency position embedding, adding frequency-related features and expanding the data to 12 channels. The principles are as follows: Where FP is the frequency position embedding vector, F is the frequency bandwidth, and i is any integer from 1 to F. The expanded data is processed using a feature extractor containing a convolutional layer and an activation function to obtain the original noisy frequency features.

6. The high-fidelity music noise reduction method based on complex environment according to claim 1, characterized in that: The specific process of step (3) is as follows: (3-1) Using the noise representation learning module based on the three-dimensional attention mechanism, the attention weight is adaptively assigned to learn and model the noise in the original noisy frequency features to obtain the noise features; (3-2) Based on the supervised attention module, feature weighting is performed to enhance the noise feature and obtain the enhanced noise feature.

7. The high-fidelity music noise reduction method based on complex environment according to claim 6, characterized in that: Step (3-2) specifically includes: The supervised attention module is used to complete feature weighting, and the noise features are convolved to generate a residual image, which is added to the original noisy frequency features for further convolution and activation to generate an attention map. The noise features are adjusted according to the attention map to generate an enhanced noise feature that only contains the features most relevant to the noise.

8. The high-fidelity music noise reduction method based on complex environment according to claim 1, characterized in that: The specific process of step (4) is: Based on the enhanced noise feature, the noise filtering module is used to remove the noise part of the original noisy audio feature and complete the feature refinement to obtain the denoised audio feature; Then, based on the inverse short-time Fourier transform, the denoised audio features are restored to the denoised high-fidelity music clips.

9. A high-fidelity music noise reduction system based on complex environments, characterized in that: The device comprises a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, the device is used to implement the high-fidelity music noise reduction method according to any one of claims 1 to 8.

Images