Digital audio noise reduction method based on time-frequency mask separation

Abstract

This invention provides a digital audio noise reduction method based on time-frequency mask separation, belonging to the field of digital signal processing technology. Through dynamic mask adjustment technology, this invention can not only adapt to changes in environmental noise in real time and optimize the noise reduction effect, but also accurately distinguish different sound sources in complex noise environments, thereby improving the speech recognition and processing capabilities of in-vehicle audio systems. By combining multimodal data acquisition and comprehensive noise modeling technology, it provides the system with more detailed noise prediction and processing basis, so as to improve the clarity and accuracy of audio processing while reducing noise interference.

Description

Technical Field

[0001] This invention relates to the field of digital signal processing technology, specifically to a digital audio noise reduction method based on time-frequency mask separation. Background Technology

[0002] Traditional audio noise reduction methods mostly rely on simple filtering techniques in the time or frequency domains, which are often ineffective in handling complex noise backgrounds, especially in multi-source noise environments. Traditional methods struggle to accurately distinguish and separate the target signal from noise. To improve noise reduction performance, researchers have attempted to use Short-Time Fourier Transform (STFT) to transform the audio signal in both time and frequency domains, generating a time-frequency spectrum. They then design time-frequency masks based on the different characteristics of the signal and noise for processing. Although this technique has made some progress, existing methods still face challenges such as time delay, high computational cost, and poor noise reduction performance when processing multi-channel audio signals in dynamic environments due to the diverse and dynamic nature of the noise.

[0003] In the prior art, publication number CN119296567A, entitled "A Method, Apparatus, Device, and Medium for Vehicle-Mounted Audio Region Separation," a multi-channel noise reduction mask is determined using a pre-trained noise reduction network. This multi-channel noise reduction mask accurately identifies the effective signal segments and noise signal segments in the echo-free vehicle-mounted array signal. Based on this multi-channel noise reduction mask, the clean audio signal, i.e., the multi-channel noise reduction array signal, can be further extracted, thereby reducing the interference of noise on subsequent audio region separation tasks and enabling the audio region separation model to more accurately identify and process audio signals. The noise reduction network decouples the audio region separation task and the noise reduction task of the audio region separation model, preventing the model from directly processing excessively noisy audio signals, which could lead to overfitting to noise features and reducing the accuracy of audio region separation.

[0004] Existing noise reduction technologies mostly passively suppress various types of noise and lack effective dynamic models to describe and predict the time-varying characteristics of these multi-source noises. In addition, traditional technologies have difficulty accurately separating speech signals in scenarios where multiple sound sources exist simultaneously, which seriously restricts the application effect of in-vehicle audio systems in speech recognition and communication.

[0005] The information disclosed in the background section above is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a digital audio noise reduction method based on time-frequency mask separation to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] The digital audio noise reduction method based on time-frequency mask separation includes the following steps:

[0009] Step S1: Multimodal data acquisition and synchronous processing:

[0010] Multimodal data includes vehicle speed signal, vehicle vibration signal, speaker feedback signal, ambient airflow signal, and door sealing parameters;

[0011] Step S2: Physical modeling of aerodynamic noise:

[0012] The vehicle speed signal and ambient airflow signal included in the multimodal data sequence are analyzed to generate a wind noise amplitude factor. Using the door sealing parameters and the wind noise amplitude factor, a dynamic prediction model of the distribution of low-frequency wind noise and high-frequency noise is established to obtain the wind noise model.

[0013] Step S3: Vehicle vibration and noise modeling:

[0014] The vehicle vibration signals included in the multimodal data sequence are processed by short-time Fourier transform to generate the time-frequency distribution of the vehicle vibration signals, which is used to describe the low-frequency resonance noise characteristics of the vehicle body and form a vibration noise model.

[0015] Step S4: Loudspeaker echo modeling:

[0016] The speaker echo signal included in the multimodal data sequence is obtained for time-frequency analysis, and combined with the door sealing parameters, an echo noise signal model is obtained.

[0017] Step S5: Construction of the noise signal fusion model:

[0018] The obtained wind noise model, vibration noise model, and echo noise signal model are used to construct a comprehensive noise distribution model.

[0019] Step S6: Initial time-frequency mask generation:

[0020] Obtain the constructed integrated noise distribution model and calculate the initial time-frequency mask;

[0021] Step S7: Perform dynamic mask optimization on the initial time-frequency mask:

[0022] The initial time-frequency mask is dynamically adjusted to generate an optimized dynamic mask.

[0023] Step S8: Multi-channel audio signal noise reduction and separation:

[0024] The generated dynamic mask is obtained, and noise reduction is performed on each channel of the multi-channel vehicle audio signal to extract the pure target audio signal. Based on the extracted target audio signal, a sound region separation operation is further performed to identify the audio regions of different sound sources.

[0025] Compared with existing technologies, the beneficial effects of this invention are: by using dynamic mask adjustment technology, it can not only adapt to changes in environmental noise in real time and optimize the noise reduction effect, but also accurately distinguish different sound sources in complex noise environments, thereby improving the speech recognition and processing capabilities of the vehicle audio system; by combining multimodal data acquisition and comprehensive noise modeling technology, it provides the system with more detailed noise prediction and processing basis, so as to improve the clarity and accuracy of audio processing while reducing noise interference. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0028] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0029] Example 1:

[0030] Please see Figure 1 The present invention provides a technical solution:

[0031] The digital audio noise reduction method based on time-frequency mask separation includes the following steps:

[0032] Step S1: Multimodal data acquisition and synchronous processing:

[0033] Multimodal data includes vehicle speed signal, vehicle vibration signal, speaker feedback signal, ambient airflow signal, and door sealing parameters;

[0034] Further explanation: Vehicle speed signal, used to analyze wind noise intensity; acquired through the vehicle's OBD interface or vehicle speed sensor;

[0035] Vehicle body vibration signals are used to analyze the distribution of low-frequency vibration noise; these signals are acquired through a built-in acceleration sensor.

[0036] The loudspeaker feedback signal is used to analyze loudspeaker echo interference; it is obtained through the loudspeaker feedback system.

[0037] Ambient airflow signals are used to capture aerodynamic noise characteristics; these are acquired through external airflow sensors on the vehicle body.

[0038] Door sealing parameters are used to reflect the impact of noise propagation; they are obtained by combining the low-frequency characteristics of vehicle body vibration signals with the noise spectrum signals collected by internal microphones.

[0039] Time alignment processing is performed on the multimodal input signal to generate a multimodal data sequence;

[0040] Further explanation: The vehicle speed signal is used to analyze wind noise intensity, specifically as follows:

[0041] The vehicle speed signal is represented as vehicle speed. The vehicle speed collected at sampling time t is recorded as v(t). Vehicle speed data (unit: km / h) is obtained through the on-board diagnostic (OBD) interface. The dynamic changes of vehicle speed v(t) at sampling time t are recorded in real time. The vehicle speed refresh frequency of the OBD interface is fixed at 10Hz to ensure the sufficiency and timeliness of data sampling.

[0042] Function of vehicle speed signal: The relationship between vehicle speed signal and wind noise intensity is defined by the following formula:

[0043]

[0044] in, The wind noise intensity at sampling time t is expressed in units of W / m².

[0045] The air density (under standard atmospheric pressure and normal temperature conditions, in this embodiment) is... (The value is taken as 1.225 kg / m³).

[0046] The vehicle's aerodynamic drag coefficient (a constant value obtained through calibration, in this embodiment) (Value is 0.28); the greater the vehicle speed v(t), the greater the corresponding wind noise intensity.

[0047] The vehicle body vibration signal is used to analyze the distribution of low-frequency vibration noise, specifically:

[0048] Using an in-vehicle acceleration sensor, triaxial vibration acceleration signals at each sampling time t near the seat on the vehicle chassis are collected, with the sampling frequency set to 1kHz; the triaxial vibration acceleration signal at each sampling time t is represented as follows: ,in These are the triaxial vibration acceleration components at sampling time t; the units are... ;

[0049] For the original triaxial vibration acceleration signal Offset calibration is performed to remove DC component interference (such as sensor-borne drift or low-frequency noise baseline interference) and improve the accuracy of subsequent frequency domain analysis of the signal.

[0050] Offset calibration is performed by averaging the time series data; the correction formula is as follows:

[0051]

[0052] N2 is the size of the sliding window used to smooth the DC component. express Any axis in the middle;

[0053] The calibrated triaxial vibration acceleration signal, and Represented as:

[0054]

[0055] Should As input for subsequent time-frequency analysis;

[0056] The role of vehicle vibration signals: Low-frequency vibrations (below 200Hz in this example) can cause resonance in the vehicle structure, which is the main source of tire noise and structural component vibration noise. Time-frequency analysis is used to extract the frequency domain characteristics of the triaxial vibration acceleration signal and calculate the total vibration noise amplitude. Specifically:

[0057] Using Short Time Fourier Transform (STFT) to Each axis is executed independently, generating a vibration acceleration signal in the corresponding frequency domain;

[0058]

[0059] Perform the following independently for each axis:

[0060]

[0061] The total vibration noise amplitude is calculated by weighted merging of the triaxial vibration acceleration signals after STFT transformation.

[0062]

[0063] in The weights are assigned to the acceleration signals of each axis, and these weights are dynamically adjusted based on the proportion of vibration energy. .

[0064] The loudspeaker feedback signal is used to analyze loudspeaker echo interference, specifically:

[0065] Obtain the speaker's feedback signal at sampling time t from the in-vehicle advanced audio playback system. ; return signal The signal includes a mixture of the speaker's direct emission signal, echo signal, and background noise; the speaker monitors its own reflection components in real time, combined with the feedback signal from the in-vehicle microphone array at sampling time t. The reflected noise is captured; specific capture methods include:

[0066] For the returned signal For noise reduction, the adaptive filtering algorithm is preferred to distinguish the echo components.

[0067] Feedback signal from the in-vehicle microphone array at sampling time t:

[0068]

[0069] in: It is a feedback signal from a multi-channel microphone array;

[0070] It is the first Each microphone channel signal;

[0071] It is the weight of each channel, which is set based on the microphone position and signal quality;

[0072] The sound signal S(t) played by the loudspeaker is used as the reference input signal;

[0073] A multi-channel microphone array is distributed throughout the vehicle to capture reflected reverberation signals;

[0074] The echo noise is extracted by jointly analyzing the return signal and the microphone array feedback signal.

[0075] The role of loudspeaker echo signal: Reflected sound or reverberation is a significant source of noise interference in the small space of a train carriage; time delay analysis is performed on the loudspeaker sound signal S(t) at sampling time t to calculate the echo delay time caused by the echo path. The echo delay time is determined by the path length from the loudspeaker to the reflector and back. The specific formula for calculating the echo delay time is as follows:

[0076]

[0077] in, t is the echo delay time at sampling time t, d is the distance from the loudspeaker to the reflecting surface; cv is the speed of sound; in this embodiment, cv is taken as 343 m / s;

[0078] Based on echo delay time The echo signal generated by the emitted sound signal S(t) on the reflection path is modeled as follows:

[0079]

[0080] in The independent echo signal formed by reflection; From The target echo signal is separated through time delay analysis and extraction algorithms;

[0081] For the time to experience echo delay The speaker emits a sound signal;

[0082] The sound absorption coefficient of the reflecting surface represents the proportion of sound wave energy reflected by the reflecting surface. ; The specific values ​​are calibrated in real time based on the feedback signal from the microphone array;

[0083] The independent echo signal is calculated and cross-compared with the multi-channel feedback signal sampled by the microphone array. Feature analysis is used to improve the recognition accuracy. A preliminary calibrated echo noise signal model is generated.

[0084]

[0085] in, This is the preliminary calibrated echo noise signal model;

[0086] The weighting of independent echo signals and microphone array feedback data is dynamically adjusted based on the real-time noise environment; initial settings ;

[0087] The ambient airflow signal is used to capture aerodynamic noise characteristics, specifically:

[0088] Airflow sensors are installed on the exterior of the vehicle (e.g., on the roof or outside the windows) to capture airflow speed and turbulence characteristics in real time during vehicle operation.

[0089] The sensor uses a high-sensitivity anemometer with a sampling frequency of 100Hz to capture the main frequency band variation characteristics of turbulence (the frequency band less than 50Hz is the main energy concentration area of ​​the environmental airflow signal).

[0090] The ambient airflow signal is represented as airflow velocity, and the airflow velocity collected at sampling time t is denoted as u(t); representing the instantaneous airflow velocity (unit: m / s) captured by the sensor at each sampling time t.

[0091] Synchronous data acquisition:

[0092] To ensure the consistency of data timing, it is synchronized with the vehicle speed acquisition module, and the sampling time points of u(t) and v(t) are aligned by timestamps;

[0093] Let N be the total number of samples corresponding to all sampling times t; the sequence data of airflow velocity u(t) is:

[0094]

[0095] Where N represents the total number of sampling points; Let u(t) represent the airflow velocity at the sampling point corresponding to the Nth sampling time t;

[0096] Low-frequency noise elimination:

[0097] The collected airflow velocity signal u(t) is cleaned using a high-pass filter to remove low-frequency input interference. The filtered airflow velocity is denoted as u'(t). The cutoff frequency of the high-pass filter is set to 2Hz. The low-frequency input interference is the mechanical noise and vibration noise of the wind speed measuring instrument.

[0098] Airflow pressure estimation:

[0099] Based on the filtered airflow velocity u'(t), the airflow pressure value is calculated using the following formula:

[0100]

[0101] in: It is the airflow pressure value per unit area (unit: Pa) corresponding to the sampling time t;

[0102] The pressure exerted by airflow on a unit area of ​​an object's surface is characterized as 1 Pa = 1 N / m². This pressure can be understood as the "dynamic pressure" generated by the fluid due to changes in velocity and density; unit area is expressed as per square meter.

[0103] It is the air density (a constant, taken as 1.225 kg / m³, under standard atmospheric pressure and normal temperature conditions).

[0104] Point-by-point calculation This forms a set of airflow pressure values ​​at different sampling times t, and the set of airflow pressure values ​​is denoted as... :

[0105]

[0106] in, In the context of signal processing, it is used as a data input source;

[0107] represent Mid-sampling time The airflow pressure value at point i is an element in the dataset; i∈{1,2,…,N};

[0108] Extraction of airflow pressure frequency domain characteristics:

[0109] right Short-Time Fourier Transform (STFT) is performed to analyze the distribution characteristics of the environmental airflow signal in the time and frequency domain, and the time and frequency distribution of the airflow pressure value is obtained. It should be noted that in digital signal processing systems, continuous signals are converted into discrete signals through sampling; this means that all calculations, including Fourier transforms, are performed at discrete points; therefore, summation is used instead of integration.

[0110]

[0111] in: It is the time-frequency distribution of airflow pressure values;

[0112] It is a Fourier transform kernel used to transform airflow pressure signals. Convert to the frequency domain;

[0113] It is the target frequency, and Perform phase rotation calculations and extract... The signal at the target frequency during the time period Amplitude and phase components on;

[0114] It is a short-time window function used to limit the time range of the Fourier transform; it defines a finite time range around the current sampling time t.

[0115] This indicates that the short-time window function affects the variable around the center of t. Scope of influence;

[0116] The value ranges from -M1 to M1, where M1 depends on the length of the window function and the length of the signal, because the signal is discretized;

[0117] Extraction results It is the dynamic time-frequency distribution of the airflow pressure signal;

[0118] Turbulent pressure amplitude normalization:

[0119] Based on the results of the Fourier transform, the time-frequency distribution in the frequency domain Normalization is performed to unify the dimensions, resulting in the normalized time-frequency distribution. ; Ensure that the airflow turbulence characteristics under different environments are comparable;

[0120] Normalization formula:

[0121]

[0122] It is the normalized time-frequency distribution matrix;

[0123] It is the maximum value of the entire time-frequency distribution matrix.

[0124] Further explanation: The door sealing parameters mentioned above are used to reflect the impact of noise propagation, specifically:

[0125] An acceleration sensor is placed on the inside of the car door to characterize the door's sealing parameters as vibration signals. The vibration signals of the door are acquired, and the vibration signal at sampling time t is recorded as follows: The sensor should have high sensitivity and be able to collect vibration characteristics in the low-frequency range, which is 0.5Hz to 10Hz.

[0126] A microphone is installed inside the vehicle to collect the time-domain signal of ambient noise inside the vehicle. (Unit: Pa);

[0127] The time-domain signal of the in-vehicle ambient noise is obtained by using Fast Fourier Transform (FFT). Convert to frequency domain signal ; It is a complex number, which includes both time-domain signals and... In frequency The amplitude information on the wave also includes the corresponding phase information;

[0128]

[0129] :frequency The corresponding signal amplitude, i.e., the amplitude;

[0130] :frequency Corresponding signal phase;

[0131] therefore, It expresses comprehensive information about the signal in the frequency dimension, including amplitude and phase;

[0132] frequency domain signal Transformed into power spectral density, which characterizes the noise power distribution The calculation formula is as follows:

[0133]

[0134] The power spectral density of the in-vehicle noise signal (unit: or ;

[0135] For frequency The corresponding amplitude;

[0136] The analysis window length (in seconds);

[0137] Acquisition of external reference noise data:

[0138] Within the same time period as the vibration signal from the car door, the time-domain signal of the external reference noise is measured, and finally, the power spectral density of the time-domain signal corresponding to the external reference noise is calculated. As a benchmark, noise collection points are set up outside the vehicle to determine the time-domain signal of the external environmental noise.

[0139] Calculate low-frequency vibration characteristics:

[0140] Based on the obtained door vibration signal ,analyze Low-frequency vibration characteristics are obtained by extracting frequency components between 0.5Hz and 10Hz using a digital filter.

[0141] Sound transmission characteristics analysis:

[0142] Calculate the acoustic transfer function of the door seal effect based on the power spectral density of the noise signals inside and outside the vehicle. :

[0143]

[0144] in, It is the sound transfer function affected by the door's sealing performance;

[0145] The power spectral density of the in-vehicle noise signal (unit: or );

[0146] The power spectral density of the external reference noise (unit: or );

[0147] Taking vibration signals into account With sound transfer function The relationship between them forms the following functional model:

[0148]

[0149] in, It is a comprehensive sealing parameter function that reflects the dual influence of time and frequency; b1 and b2 are empirical adjustment coefficients, b1+b2=1, and the values ​​of b1 and b2 are both within the interval (0,1); in this embodiment, based on experience, the initial values ​​of b1 and b2 are both set to 0.5; the weights of b1 and b2 are determined by the entropy weight method and the fuzzy hierarchical analysis method (FAHP).

[0150] The multimodal input signal is time-aligned to generate a multimodal data sequence, which is achieved through interpolation and timing calibration.

[0151] In this embodiment, the multimodal data sequence is characterized as follows:

[0152]

[0153] It is a vector that integrates all multimodal signals at sampling time t;

[0154] Interpolation processing: Interpolation is used to unify all modal signals into a data format with the same time step; the highest global sampling frequency is determined and used as the target reference step size for interpolation;

[0155] Interpolation is performed on the low-frequency sampled signal: linear interpolation is used to smooth the transition of the low-frequency signal and generate a continuous data sequence with equal time intervals.

[0156] For highly nonlinear dynamic segments, accuracy can be enhanced by using higher-order interpolation methods (such as cubic spline interpolation).

[0157] Timing calibration: Based on the key event sensitivity of vehicle speed signal change characteristics (such as acceleration, deceleration, constant speed, etc.), the timing of other vehicle vibration signals, speaker feedback signals, and ambient airflow signals is dynamically adjusted to be completely aligned with the vehicle speed signal time axis; the calibration steps include:

[0158] Select vehicle speed signal as the time base:

[0159] Since vehicle speed is an important factor affecting wind noise and is highly correlated with changes in external airflow signals and vehicle vibration signals, vehicle speed signals are selected as the main reference axis for time calibration.

[0160] Dynamically adjust the sampling time axis of other modal signals:

[0161] Extract the points of change in the vehicle speed signal (such as the inflection points of speed increase or decrease) as time synchronization points;

[0162] Based on these time synchronization points, the vehicle vibration signal, speaker feedback signal, and ambient airflow signal are aligned using linear interpolation and time offset compensation.

[0163] Ensure time synchronization accuracy:

[0164] Verify the error of the calibrated data, such as whether the time difference of the main modal signals of the model is within the allowable range (e.g., less than 1 ms).

[0165] After processing, the generated multimodal data sequence has the following characteristics:

[0166] Unified time base:

[0167] The time axes of all modal signals are fully aligned after interpolation and calibration, with consistent time steps, meeting the requirements for dynamic wind noise analysis.

[0168] Coupling dynamic characteristics:

[0169] Time alignment allows the dynamic characteristics of different modal signals to be correlated (such as the response of vehicle speed changes to airflow pressure fluctuations and low-frequency vibration noise), providing accurate input data for subsequent dynamic prediction models.

[0170] Continuity and smoothness:

[0171] Interpolation eliminates the time gap problem of low-frequency sampling signals, and calibration ensures the time consistency of multimodal data.

[0172] Ultimately, the multimodal data sequence possesses complete temporal synchronization characteristics, which can meet the input requirements of the wind noise dynamic prediction model.

[0173] Step S2: Physical modeling of aerodynamic noise:

[0174] The vehicle speed signal and ambient airflow signal included in the multimodal data sequence are analyzed to generate a wind noise amplitude factor. Using the door sealing parameters and the wind noise amplitude factor, a dynamic prediction model of the distribution of low-frequency wind noise and high-frequency noise is established to obtain the wind noise model.

[0175] To further explain, the modeling steps for the wind noise model are as follows:

[0176] Calculate the relationship between the vehicle speed signal and the corresponding airflow characteristics of the ambient airflow signal to generate the wind noise amplitude factor;

[0177] By utilizing the door sealing parameters and combining them with the wind noise amplitude factor, a dynamic prediction model for the distribution of low-frequency wind noise and high-frequency noise is established, thus obtaining the wind noise model; the acquisition of the wind noise model is specifically as follows:

[0178] The wind noise amplitude factor is defined to characterize the dynamic proportional relationship between air pressure and the intensity of wind noise caused by vehicle speed. The wind noise amplitude factor at sampling time t is denoted as... , The formula is as follows:

[0179]

[0180] in, Characterizes the airflow pressure per unit area at sampling time t;

[0181] Characterizes the wind noise intensity caused by vehicle speed at sampling time t;

[0182] Combining the wind noise amplitude factor, the following wind noise model is obtained:

[0183]

[0184] in, It is the normalized time-frequency distribution; It is a comprehensive sealing parameter function. A dynamic predictive quantification model representing the distribution of wind noise intensity is used to analyze and evaluate the dynamic characteristics of wind noise distribution in time series and frequency range;

[0185] It is the frequency dimension, used to describe the distribution characteristics of wind noise in the frequency domain. The distribution characteristics include the characteristics of low-frequency wind noise and high-frequency noise.

[0186] This dynamic prediction model can reflect the wind noise characteristics under the combined effect of airflow pressure, vehicle speed and door sealing parameters in real time.

[0187] Dynamic wind noise amplitude factor It closely resembles real-world operating conditions and can adjust in real time based on changes in airflow pressure, vehicle speed, and door sealing parameters, thus improving the practical value and robustness of wind noise modeling.

[0188] Step S3: Vehicle vibration and noise modeling:

[0189] The vehicle vibration signals included in the multimodal data sequence are processed by short-time Fourier transform (STFT) to generate the time-frequency distribution of the vehicle vibration signals, which is used to describe the low-frequency resonance noise characteristics of the vehicle body and form a vibration noise model.

[0190] Further explanation: Regarding the total vibration noise amplitude Low-pass filtering is performed to remove high-frequency interference; a Butterworth filter (2nd order) is used, with a cutoff frequency set to [value missing]. The total low-frequency vibration amplitude after filtering is denoted as: ;

[0191]

[0192] in: It is the total low-frequency vibration amplitude after filtering;

[0193] : Indicates low-pass filter operation;

[0194] The filtered vibration amplitude distribution is normalized to obtain the following vibration noise model:

[0195]

[0196] in: It is the time-frequency distribution of vibration noise corresponding to the normalized vibration noise model; the value takes the range [0,1].

[0197] It is the maximum value in the entire time-frequency matrix.

[0198] Step S4: Loudspeaker echo modeling:

[0199] The speaker echo signal included in the multimodal data sequence is obtained for time-frequency analysis, and combined with the door sealing parameters, an echo noise signal model is obtained.

[0200] Further explanation: Combining the door sealing parameters, the echo noise signal model is obtained, specifically including:

[0201] Comprehensive sealing parameter function Compared with the preliminary calibrated echo noise signal model By combining these parameters, the following echo noise signal model is obtained:

[0202]

[0203] in, It is the final calibrated echo noise signal model.

[0204] Step S5: Construction of the noise signal fusion model:

[0205] The obtained wind noise model, vibration noise model, and echo noise signal model are used to construct a comprehensive noise distribution model.

[0206] Further explanation regarding the wind noise model Vibration and noise model and echo noise signal model Frequency resampling is performed; the purpose of resampling is to unify the frequency dimension of each model, so that... Specifically:

[0207] Each model signal is transformed from the time domain to the frequency domain using Fast Fourier Transform (FFT);

[0208] Select a uniform sampling frequency and frequency range Frequency resampling is performed on each model;

[0209] Use the same frequency index for each model in the frequency domain. Ensure that the frequency scale and frequency resolution are consistent; index the same frequency. The wind noise model, vibration noise model, and echo noise signal model are denoted as follows: , and ;

[0210] The integrated noise distribution model is characterized as follows:

[0211]

[0212] in, It is a comprehensive noise distribution model; It is a wind noise model. It is a vibration noise model. It is an echo noise signal model; t and These represent the time and frequency dimensions of the signal, respectively.

[0213] These are weight parameters; they can be adaptively adjusted according to the application scenario through optimization algorithms, or the corresponding weights can be determined through entropy weighting and fuzzy hierarchical analysis (FAHP).

[0214] Step S6: Initial time-frequency mask generation:

[0215] Obtain the constructed integrated noise distribution model and calculate the initial time-frequency mask;

[0216] Further explanation: Let the initial time-frequency mask be denoted as... , The expression is determined as follows:

[0217]

[0218] in, The time-frequency distribution of the input audio signal. This is a comprehensive noise distribution model.

[0219] Step S7: Perform dynamic mask optimization on the initial time-frequency mask:

[0220] Based on the changing characteristics of the real-time noise environment, the initial time-frequency mask is dynamically adjusted to generate an optimized dynamic mask;

[0221] Further explanation: The optimized dynamic mask is denoted as... ;

[0222]

[0223] in, It represents the time-varying derivative of the integrated noise distribution model, reflecting the dynamic characteristics of the noise; This is a balancing factor that controls the weight between the dynamic mask and the time-varying derivative.

[0224] Step S8: Multi-channel audio signal noise reduction and separation:

[0225] The generated dynamic mask is obtained, and noise reduction is performed on each channel of the multi-channel vehicle audio signal to extract the pure target audio signal. Based on the extracted target audio signal, a sound region separation operation is further performed to identify the audio regions of different sound sources.

[0226] Further explanation: Multi-channel audio signal noise reduction and separation, specifically including:

[0227] Get the optimized dynamic mask ;

[0228] Signal decomposition and time-frequency conversion:

[0229] Let the time-frequency distribution of the input audio signal of the m-th channel be denoted as... , , This represents the total number of channels; the audio signal for each channel is converted to a time-frequency representation using a Short-Time Fourier Transform (STFT).

[0230]

[0231] in, It is the time-domain signal of the m-th channel. This indicates the short-time Fourier transform operation;

[0232] Apply dynamic mask:

[0233] For each time frequency point Calculate the separation weights for noise and target signal; apply dynamic masking. Time-frequency distribution applied to each channel representation Obtain the signal after noise suppression. :

[0234]

[0235] in, This represents the time-frequency signal of the m-th channel after noise reduction; when applying a dynamic mask:

[0236] If the noise level is higher, then The larger the dynamic mask, the better. The closer a value is to 0, the stronger the noise suppression.

[0237] If the target signal is stronger, then Dynamic mask The closer a value is to 1, the more completely the target signal is preserved.

[0238] When the noise distribution changes drastically... Provides additional dynamic adjustment capabilities to improve the real-time performance and adaptability of the mask;

[0239] The inverse Fourier transform (ISTFT) recovers the signal, specifically:

[0240] Inverse transformation of the time-frequency signal into a time-domain signal:

[0241] For the time-frequency signal of the m-th channel after noise reduction Perform inverse short-time Fourier transform (ISTFT) to recover the denoised time-domain signal. :

[0242]

[0243] in, This indicates the inverse short-time Fourier transform operation. This represents the time-domain signal of the m-th channel after dynamic mask denoising.

[0244] After noise reduction of the multi-channel audio signal, dynamic masking is used to separate the audio sources in the denoised signal to identify the audio regions of different sound sources. The specific steps are as follows:

[0245] Time-frequency feature extraction:

[0246] Time-frequency analysis is performed on the time-domain signal of the m-th channel after noise reduction to extract the time-frequency feature representation of each channel:

[0247]

[0248] Sound source separation based on time-frequency masking:

[0249] Time-frequency representation for each audio channel By applying a sound source separation algorithm, components from different sound sources are separated from the multi-channel signal; this step utilizes dynamic masking to separate the spectral characteristics of different sound sources.

[0250] The sound source separation algorithm can be a clustering algorithm or a deep learning model;

[0251] For example, clustering algorithms (such as K-means) can be used to separate the signal in each channel of the time-frequency domain, resulting in an independent time-frequency representation of each sound source:

[0252]

[0253] in, This represents a clustering algorithm that transforms the denoised time-frequency representation... Divided into different sound source components .

[0254] Inverse Fourier transform recovers the audio source signal:

[0255] For each separated sound source signal Perform inverse short-time Fourier transform to recover the time-domain signals of each sound source. :

[0256]

[0257] In this way, multiple sound sources are effectively separated and recovered into independent time-domain signals.

[0258] Output the noise-reduced audio signal:

[0259] Finally, output the noise-reduced and separated audio signals from all channels. It serves as a clean audio signal for applications such as voice recognition and noise suppression in in-vehicle systems.

[0260] The beneficial effects of this embodiment are as follows:

[0261] This technical solution utilizes time-frequency masking to reduce noise and separate sound sources in multi-channel audio signals, accurately capturing the time-frequency characteristics of noise and target signals.

[0262] By dynamically adjusting the time-frequency mask, it can adapt to changes in environmental noise in real time and optimize the noise reduction effect.

[0263] Using time-frequency masks to effectively separate multiple audio sources improves the speech recognition and audio processing capabilities of in-vehicle audio systems in complex noise environments.

[0264] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0265] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0266] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0267] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0268] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0269] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A digital audio noise reduction method based on time-frequency mask separation, characterized in that, The specific steps include: Step S1: Multimodal data acquisition and synchronization processing: The multimodal data includes vehicle speed signal, vehicle body vibration signal, loudspeaker echo signal, environmental airflow signal and vehicle door sealing parameter; Step S2: Aerodynamic noise physical modeling: The vehicle speed signal and environmental airflow signal included in the multimodal data sequence are analyzed to generate a wind noise amplitude factor, and a dynamic prediction model of low-frequency wind noise and high-frequency noise distribution is established by combining the wind noise amplitude factor with the vehicle door sealing parameter, to obtain a wind noise model; Step S3: Vehicle body vibration noise modeling: The vehicle body vibration signal included in the multimodal data sequence is processed by short-time Fourier transform to generate a time-frequency distribution of the vehicle body vibration signal, which is used to describe the low-frequency resonance noise characteristics of the vehicle body and form a vibration noise model; Step S4: Loudspeaker echo modeling: The loudspeaker echo signal included in the multimodal data sequence is subjected to time-frequency analysis, and the vehicle door sealing parameter is combined to obtain an echo noise signal model; Step S5: Noise signal fusion model construction: The wind noise model, vibration noise model and echo noise signal model obtained are used to construct a comprehensive noise distribution model; Step S6: Initial time-frequency mask generation: The comprehensive noise distribution model constructed is used to calculate an initial time-frequency mask; Step S7: Dynamic mask optimization for the initial time-frequency mask: The initial time-frequency mask is dynamically adjusted to generate an optimized dynamic mask; Step S8: Multi-channel audio signal noise reduction and separation: The generated dynamic mask is used to perform noise reduction processing on the multi-channel vehicle audio signal channel by channel to extract pure target audio signals; based on the extracted target audio signals, further audio region separation operations are performed to identify audio regions of different sound sources.

2. The time-frequency mask separation based digital audio noise reduction method of claim 1, wherein: The vehicle speed signal is used to analyze wind noise intensity; The vehicle body vibration signal is used to analyze low-frequency vibration noise distribution; The loudspeaker echo signal is used to analyze loudspeaker echo interference; The environmental airflow signal is used to capture aerodynamic noise characteristics; The vehicle door sealing parameter is used to reflect noise propagation influence; The multimodal input signals are subjected to time alignment processing to generate a multimodal data sequence.

3. The time-frequency mask separation based digital audio noise reduction method of claim 2, wherein: The vehicle speed signal is used to analyze wind noise intensity, specifically: The vehicle speed signal is represented as vehicle speed, and the vehicle speed collected at sampling time t is denoted as v(t); The correlation formula between the vehicle speed signal and wind noise intensity is defined as: wherein represents the wind noise intensity at the sampling time t; For air density; C D is the aerodynamic drag coefficient of the vehicle; the greater the vehicle speed v(t) is, the greater the corresponding wind noise intensity is; The vehicle body vibration signal is used to analyze low-frequency vibration noise distribution, specifically: The three-axis vibration acceleration signals of the collection vehicle body chassis near the seat at each sampling time t are collected, and the three-axis vibration acceleration signals at each sampling time t are represented as wherein are three-axis vibration acceleration components at the sampling time t, respectively. The frequency domain features of the triaxial vibration acceleration signals are extracted using time-frequency analysis, and the total vibration noise amplitude is calculated ; The loudspeaker echo signal is used to analyze loudspeaker echo interference, specifically: acquiring the return signal of the loudspeaker at the sampling time t ; return signal a mixed signal containing the direct emission signal of the loudspeaker, the echo signal and the background noise; the feedback signal of the in-vehicle microphone array at the sampling time t , capturing the reflected noise; the specific capturing method includes: delaying the loudspeaker sound signal S(t) at the sampling time t and calculating the echo delay time caused by the echo path ; According to echo delay time The echo signal resulting from the sound-emitting signal S(t) on the reflection path is modeled by the expression wherein is the reflected, independent echo signal; to experience echo delay time of the loudspeaker sound signal; The sound absorption coefficient of the reflection surface indicates the proportion of energy reflected by the reflection surface to the sound wave; ; The independent echo signal calculation is cross-compared with the multi-channel feedback signal sampled by the microphone array to preliminarily generate a calibrated echo noise signal model: wherein is the echo noise signal model after preliminary calibration; To independently echo signal and microphone array feedback data weight proportion, the ambient airflow signal is characterized as airflow velocity, and the airflow velocity collected at the sampling time t is denoted as u(t); the total number of samples corresponding to all sampling times t is denoted as N; A high-pass filter is used to perform basic cleaning on the collected airflow speed signal u(t) to filter out low-frequency input interference, and the filtered airflow speed is denoted as u'(t); The airflow pressure value is calculated using the following formula based on the filtered airflow speed u'(t): wherein: is the air flow pressure value per unit area corresponding to the sampling time t; is the air density; point-by-point calculation , a set of airflow pressure values at different sampling instants t is formed and denoted as : wherein representing the sampling time instant the airflow pressure value at the sampling time instant, is an element of the dataset; i ∈ {1, 2, …, N}; To perform short-time Fourier transform, analyze the distribution characteristics of the environmental airflow signal in the time-frequency domain, and obtain the time-frequency distribution of the airflow pressure value ; wherein is the target frequency; Based on the result of Fourier transform, the time-frequency distribution in frequency domain is obtained The normalized time-frequency distribution is obtained by normalization and dimensionless .

4. The time-frequency mask separation based digital audio noise reduction method of claim 3, wherein: The vehicle door sealing parameter is used to reflect noise propagation influence, specifically: The door sealing performance parameter is characterized as a vibration signal of the door, the vibration signal of the door is acquired, and the vibration signal at a sampling time t is denoted as ; Collecting time domain signals of environmental noise in a vehicle ; The time-domain signal of the in-vehicle ambient noise is obtained by using Fast Fourier Transform. Convert to frequency domain signal ; The frequency domain signal is converted into a power spectral density characterizing the noise power distribution The calculation formula is as follows: is the power spectral density of the in-car noise signal; for frequency corresponding amplitude to analyze the window length; In the same time period to which the vibration signal of the vehicle door belongs, a time domain signal of the outside reference noise of the vehicle is measured, and finally the power spectrum density of the time domain signal corresponding to the outside reference noise of the vehicle is obtained as a contrast reference; According to the obtained vehicle door vibration signal , analyze Low-frequency vibration characteristics, extract frequency components between 0.5 Hz and 10 Hz through a digital filter; According to the power spectral density of the in-car and out-car noise signals, the sound transmission function of the door seal influence is calculated : wherein is the sound transfer function of the door sealing influence; is the power spectral density of the in-car noise signal; is the power spectral density of the off-board reference noise; Taking into account the relationship of the vibration signal with the acoustic transfer function the following function model is formed: wherein, is a comprehensive sealing parameter function reflecting the dual influence of time and frequency; b1 and b2 are empirical adjustment coefficients, b1+b2=1, and b1 and b2 are both in the interval (0,1).

5. The time-frequency mask separation based digital audio noise reduction method of claim 4, wherein: The modeling steps of the wind noise model are as follows: A relationship between a vehicle speed signal and an airflow characteristic corresponding to an ambient airflow signal is calculated to generate a wind noise amplitude factor; A dynamic prediction model of low-frequency wind noise and high-frequency noise distribution is established by using the door sealing parameter and the wind noise amplitude factor to obtain a wind noise model; the wind noise model is obtained in the following manner: The wind noise amplitude factor is defined to represent the dynamic proportional relationship between the air flow pressure value and the wind noise intensity caused by the vehicle speed, and the wind noise amplitude factor at the sampling time t is denoted as , The formula is as follows: wherein, characterizes the pressure value of the gas flow per unit area at the sampling instant t; characterizing the wind noise intensity at the sampling instant t due to the vehicle speed; The wind noise model is obtained in the following manner by using the wind noise amplitude factor: wherein, is the normalized time-frequency distribution; is the integrated sealing parameter function, represents a dynamic prediction quantization model of wind noise intensity distribution, used for analyzing and evaluating the dynamic characteristics of the distribution of wind noise in the time series and frequency range; is a frequency dimension, used to describe the distribution characteristics of the wind noise in the frequency domain, the distribution characteristics including the characteristics of low-frequency wind noise and high-frequency noise.

6. The time-frequency mask separation based digital audio noise reduction method of claim 5, wherein: The total vibration noise amplitude is calculated as follows: The total vibration amplitude is low-pass filtered, and the filtered low-frequency total vibration amplitude is denoted as The filtered vibration amplitude distribution is normalized to obtain the following vibration noise model: wherein: is the normalized vibration noise model corresponding to the vibration noise time-frequency distribution; is the maximum value in the entire time-frequency matrix.

7. The time-frequency mask separation based digital audio noise reduction method of claim 6, wherein: An echo noise signal model is obtained by using the door sealing parameter, and specifically includes: combining the integrated sealing parameter function with the preliminary calibrated echo noise signal model to obtain the following echo noise signal model: wherein, is the final calibrated echo noise signal model.

8. The time-frequency mask separation based digital audio noise reduction method of claim 7, wherein: wind noise model vibration noise model and echo noise signal model performing frequency resampling; Each model signal is converted from a time domain to a frequency domain by using a fast Fourier transform; Selecting a uniform sampling frequency and frequency range Frequency resampling each model; so that each model has a consistent frequency scale and frequency resolution in the frequency domain; and 、 and ; The comprehensive noise distribution model is represented in the following manner: wherein, is a comprehensive noise distribution model; is a wind noise model, is a vibration noise model, is an echo noise signal model; t and denote the time and frequency dimensions of the signal, respectively; is a weight parameter.

9. The time-frequency mask separation based digital audio noise reduction method of claim 8, wherein: Let the initial time-frequency mask be denoted by , The expression is determined as follows: wherein is a time-frequency distribution of the input audio signal, is a comprehensive noise distribution model; Let the optimized dynamic mask be denoted as ; wherein, denotes a time-varying derivative of the comprehensive noise distribution model, reflecting the dynamic characteristics of the noise; is a balance factor, controlling the weight between the dynamic mask and the time-varying derivative.

10. The time-frequency mask separation based digital audio noise reduction method of claim 9, wherein: The multi-channel audio signal noise reduction and separation specifically includes: Obtaining an optimized dynamic mask ; let the time-frequency distribution of the mth channel input audio signal be denoted as , , is the total number of channels; convert the audio signal of each channel into a time-frequency representation using a short-time Fourier transform: wherein is the time domain signal of the mth channel, denotes a short-time Fourier transform operation; For each time-frequency bin , compute separation weights for noise and target signal; apply dynamic mask to time-frequency distribution of each channel representation , obtain noise-suppressed signal : wherein, is the time-frequency signal of the mth channel after noise reduction; when a dynamic mask is applied: The higher the noise level is, the The dynamic mask Tends to 0, the stronger the noise suppression is. If the target signal is stronger, then Dynamic mask The closer a value is to 1, the more completely the target signal is preserved. time-frequency signal of the mth channel after noise reduction performing inverse short-time Fourier transform to restore time-domain signal after noise reduction : wherein, denotes an inverse short-time Fourier transform operation, is the time-domain signal of the m-th channel after noise reduction based on dynamic masking. After the multi-channel audio signal noise reduction is completed, a dynamic mask is used to separate the noise-reduced signal to identify different audio regions of different sound sources.

Images