A method and system for acoustic signal denoising based on feature fusion

Abstract

The present invention discloses a method and system for reducing noise of acoustic signals based on feature fusion, the method comprising: S1: collecting measured noise signals and constructing simulated target signals; S2: constructing feature vectors of the measured noise signals and the simulated target signals based on the features of the measured noise signals and the simulated target signals respectively; S3: constructing a fused feature vector of the simulated acoustic signal based on the feature vectors of the measured noise signals and the simulated target signals; S4: inputting the fused feature vector of the simulated acoustic signal into a multilayer perceptron for training to obtain a trained multilayer perceptron model; S5: performing transfer learning on the trained multilayer perceptron model to achieve denoising of the measured acoustic signals. The present invention adopts measured acoustic signals and simulated target signals to achieve effective training of the multilayer perceptron; through transfer learning of the multilayer perceptron model, when the measured target signal cannot be obtained, the noise reduction processing of the measured acoustic signal is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of acoustic signal denoising, and in particular to an acoustic signal denoising method and system based on feature fusion. Background Art

[0002] During the transmission of sound signals, noise interference is caused by the natural environment, human activities or other electronic devices; the noise interference in the sound signal will affect people's auditory experience and will also have a negative impact on the fields of communication and audio recognition. Denoising the sound signal is conducive to improving the quality of the sound signal, obtaining a clean target signal, and further analyzing and processing the target. Currently, common sound signal denoising methods include filtering methods based on the time and frequency domains and target signal extraction methods based on machine learning. Among them, the filtering method is to perform signal processing on the data of the time and frequency domains of the sound signal, such as mean filtering, median filtering, and hybrid filtering; the machine learning method is a relatively new denoising method, which mainly trains the machine learning model by learning the mapping relationship between the input noisy sound signal and the noise-free target signal, thereby achieving noise reduction processing of the sound signal.

[0003] Machine learning methods have shown advantages over filtering methods in terms of acoustic signal denoising. They have good denoising effects, can process a large number of acoustic signals in real time, and have high adaptability to complex environments. However, machine learning methods are dependent on training data. The training of machine learning models requires obtaining a large number of noisy acoustic signals and clean target signals under actual measurement conditions. Due to the limitations of the environment and sensor equipment, it is currently impossible to obtain clean target signals under actual measurement conditions; this greatly affects the acoustic signal denoising performance based on machine learning models. Summary of the invention

[0004] In view of this, the present invention simulates and obtains a simulated target signal under laboratory conditions, constructs a fusion feature vector of the simulated sound signal based on the measured sound signal and the simulated target signal, trains a machine learning model using the fusion feature vector of the simulated sound signal, and then performs transfer learning on the model to achieve noise reduction processing of the measured sound signal.

[0005] The technical solution proposed in the present invention is a method for reducing noise of acoustic signals based on feature fusion, comprising the following steps:

[0006] S1: Collect the measured noise signal and construct the simulated target signal, including:

[0007] The audio sensor is used to collect the acoustic signal in the case of no target, and the acoustic signal in the case of no target is marked as the measured noise signal. ;

[0008] Use simulation software to construct a speech signal and mark the constructed speech signal as a simulated target signal ;

[0009] S2: constructing feature vectors of the measured noise signal and the simulated target signal based on the features of the measured noise signal and the simulated target signal, respectively, including:

[0010] S21: Normalize the measured noise signal and the simulated target signal respectively:

[0011] ;

[0012] ;

[0013] in, represents the normalized measured noise signal, represents the normalized simulated target signal, Represents the measured noise signal The minimum value of Represents the measured noise signal The maximum value of Represents the analog target signal The minimum value of Represents the analog target signal The maximum value of

[0014] S22: The normalized measured noise signal and the normalized simulated target signal Input into the long short-term memory network respectively, and get The time domain characteristics of and The time domain characteristics of ;

[0015] S23: The normalized measured noise signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured noise signal ;

[0016] The normalized simulated target signal Perform short-time Fourier transform to obtain the frequency domain signal of the simulated target signal ;

[0017] The frequency domain signal of the measured noise signal is and the frequency domain signal of the simulated target signal Input convolutional neural network and get Frequency domain characteristics and Frequency domain characteristics ;

[0018] S24: The time domain characteristics of the measured noise signal Frequency domain characteristics Splice and get the characteristic vector of the measured noise signal :

[0019] ;

[0020] The time domain characteristics of the simulated target signal Frequency domain characteristics Splice to get the characteristic vector of the simulated target signal :

[0021] ;

[0022] S3: constructing a fusion feature vector of the simulated acoustic signal based on the feature vectors of the measured noise signal and the simulated target signal;

[0023] S4: inputting the fused feature vector of the simulated sound signal into the multi-layer perceptron for training to obtain a trained multi-layer perceptron model;

[0024] S5: Perform transfer learning on the trained multi-layer perceptron model to achieve denoising of measured sound signals.

[0025] Optionally, in step S3, constructing a fusion feature vector of the simulated acoustic signal based on the feature vectors of the measured noise signal and the simulated target signal includes:

[0026] S31: and Expand one dimension respectively, and then add the expanded dimensions and Perform vector product operation to obtain the fusion matrix of the simulated sound signal , ,in, express and Original dimension, express OK A real matrix of columns, Representation Matrix belong OK A real matrix of columns;

[0027] S32: Fusion matrix for analog acoustic signals Perform dimensionality reduction to obtain the fusion feature vector of the simulated acoustic signal :

[0028] ;

[0029] in, represents the constructed weight matrix, express The transposed matrix of .

[0030] Optionally, in step S32, the weight matrix The construction method includes:

[0031] Constructing the weight matrix The objective function is:

[0032] ;

[0033] in, Represents the weight matrix The objective function is represents the target inter-class scatter matrix, Represents the intra-class scatter matrix of targets of the same category;

[0034] The objective function The solution of is transformed into an eigenvalue solution:

[0035] ;

[0036] in, represents the feature vector, represents the eigenvalue;

[0037] Before The feature vectors form a weight matrix .

[0038] Optionally, in step S4, the fused feature vector of the simulated acoustic signal is input into a multi-layer perceptron for training to obtain a trained multi-layer perceptron model, including:

[0039] S41: Input the fused feature vector of the simulated sound signal into the multi-layer perceptron to obtain the output mapping:

[0040] ;

[0041] in, represents the output mapping, represents the multi-layer perceptron model parameters, Indicates that based on parameters The constructed multi-layer perceptron model function;

[0042] S42: Based on the output mapping and the simulated target signal obtained in step S1, define the loss function of the multilayer perceptron:

[0043] ;

[0044] in, Represents the loss function of the multi-layer perceptron;

[0045] S43: Using the back propagation algorithm to calculate the loss function of the multilayer perceptron Multilayer Perceptron model parameters The gradient of the multilayer perceptron model is updated using the stochastic gradient descent method ;

[0046] S44: Repeat steps S41, S42 and S43 until the loss function Convergence; get the trained multi-layer perceptron model.

[0047] Optionally, in step S5, transfer learning is performed on the trained multi-layer perceptron model to implement denoising of the measured sound signal, including:

[0048] S51: Extraction of measured sound signals The fused feature vector of dimension ;

[0049] S52: The fusion feature vector of the measured sound signal Input the trained multi-layer perceptron model, fine-tune the model parameters, and the model output is the denoised measured sound signal.

[0050] Optionally, in step S51, the measured sound signal is extracted The fused feature vector of dimension ,include:

[0051] S511: Normalize the measured sound signal:

[0052] ;

[0053] in, represents the normalized measured sound signal, Indicates the measured sound signal The minimum value of Indicates the measured sound signal The maximum value of

[0054] S512: The normalized measured sound signal Input long short-term memory network, get The time domain characteristics of ;

[0055] S513: The normalized measured sound signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured sound signal ;

[0056] The frequency domain signal of the measured sound signal Input convolutional neural network and get Frequency domain characteristics ;

[0057] S514: Time domain features Frequency domain characteristics Splice to obtain the fusion feature vector of the measured sound signal :

[0058] .

[0059] The present invention also provides an acoustic signal denoising system based on feature fusion, comprising:

[0060] Signal acquisition building module: sensor acquisition signal, acquisition of measured noise signal; simulation of speech signal, construction of simulated target signal;

[0061] Feature extraction module: signal normalization, extraction of time domain features, extraction of frequency domain features, and concatenation of features to obtain feature vectors;

[0062] Feature fusion module: expands the dimension of feature vectors, performs vector product operations, and fuses matrix dimensionality reduction;

[0063] Multilayer perceptron training module: multilayer perceptron output mapping, definition of multilayer perceptron loss function, and update of multilayer perceptron model parameters;

[0064] Measured sound signal denoising module: extracts the fusion feature vector of the measured sound signal and fine-tunes the multi-layer perceptron model parameters.

[0065] Beneficial effects:

[0066] The present invention extracts features based on measured sound signals and simulated target signals respectively, performs feature fusion on the basis of considering the interactivity and correlation between features, simulates the scenario of clean target signals propagating in different types of noise environments, is conducive to obtaining the expression of simulated target signals in different types of measured noise signal spaces, and constructs fused feature vectors; solves the problem of difficulty in obtaining clean target signals under measured conditions, and solves the problem of the source of training data for multi-layer perceptrons, and realizes effective training of multi-layer perceptrons; on this basis, through the transfer learning of the multi-layer perceptron model, the noise reduction processing of the measured sound signals is realized when the measured target signals cannot be obtained.

[0067] In the process of respectively extracting the features of the measured noise signal and the simulated target signal, the present invention normalizes the signal, which is beneficial to unifying the signal amplitude and improving the applicability of the method to different signals; extracting the time domain and frequency domain features of the measured noise signal and the simulated target signal is beneficial to fully extracting the signal features; in the feature fusion process, the feature vector to be fused is expanded by one dimension, and then the vector product is performed to obtain the fusion matrix, which is beneficial to retaining the information of the feature vector and can also make full use of the correlation information between the feature vectors; and then the weight matrix is ​​used to reduce the dimension of the fusion matrix, which is beneficial to reducing the dimension of the fusion matrix and improving the system processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A schematic flow chart of a method for acoustic signal denoising based on feature fusion provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0070] Embodiment 1:

[0071] A method for acoustic signal denoising based on feature fusion, such as Figure 1 As shown, the following steps are included:

[0072] S1: Collect the measured noise signal and construct the simulated target signal:

[0073] The audio sensor is used to collect the acoustic signal in the case of no target, and the acoustic signal in the case of no target is marked as the measured noise signal. ;

[0074] Use simulation software to construct a speech signal and mark the constructed speech signal as a simulated target signal .

[0075] Training a machine learning model requires a large number of noise-free target signals; however, in actual environments, due to signal propagation noise, environmental noise, multipath propagation, and sensor noise, it is impossible to collect noise-free target signals; therefore, the present invention uses simulation software to construct simulated target signals; the constructed simulated target signals can ensure that there is no noise interference.

[0076] In this embodiment, a simulated target signal can be constructed in MATLAB software, and the specific steps include:

[0077] 1. Define signal frequency, period, sampling rate, propagation medium parameters, and simulate sine wave signals;

[0078] 2. Set the sine wave signal as the transmitting signal;

[0079] 3. Set the distance, speed, acceleration and orientation parameters of the simulated target;

[0080] 4. Calculate the echo delay of the transmitted signal and construct the voice signal, that is, simulate the target signal.

[0081] You can also build a simulated target signal in the professional acoustic simulation software COMSOL Multiphysics. The specific steps include:

[0082] 1. Create a new model and select Time Domain Signal in the Physics Interface.

[0083] 2. Create a geometric model of the target scene, set the position of the simulated target, the material type of the simulated target (such as speech signal absorption coefficient, speech signal reflection coefficient), and the boundary conditions of the simulated target (such as complete reflection, partial reflection and absorption);

[0084] 3. Set up the solver to obtain the speech signal, which is the simulated target signal.

[0085] S2: Construct feature vectors of the measured noise signal and the simulated target signal based on the features of the measured noise signal and the simulated target signal respectively:

[0086] S21: Normalize the measured noise signal and the simulated target signal respectively:

[0087] ;

[0088] ;

[0089] in, represents the normalized measured noise signal, represents the normalized simulated target signal, Represents the measured noise signal The minimum value of Represents the measured noise signal The maximum value of Represents the analog target signal The minimum value of Represents the analog target signal The maximum value of

[0090] S22: The normalized measured noise signal and the normalized simulated target signal Input into the long short-term memory network respectively, and get The time domain characteristics of and The time domain characteristics of ;

[0091] S23: The normalized measured noise signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured noise signal ;

[0092] The normalized simulated target signal Perform short-time Fourier transform to obtain the frequency domain signal of the simulated target signal ;

[0093] The frequency domain signal of the measured noise signal is and the frequency domain signal of the simulated target signal Input convolutional neural network and get Frequency domain characteristics and Frequency domain characteristics ;

[0094] S24: The time domain characteristics of the measured noise signal Frequency domain characteristics Splice and get the characteristic vector of the measured noise signal :

[0095] ;

[0096] The time domain characteristics of the simulated target signal Frequency domain characteristics Splice to get the characteristic vector of the simulated target signal :

[0097] .

[0098] S3: Based on the feature vectors of the measured noise signal and the simulated target signal, a fusion feature vector of the simulated acoustic signal is constructed:

[0099] S31: and Expand one dimension respectively, and then add the expanded dimensions and Perform vector product operation to obtain the fusion matrix of the simulated sound signal , ,in, express and Original dimension, express OK A real matrix of columns, Representation Matrix belong OK A real matrix of columns;

[0100] S32: Fusion matrix for analog acoustic signals Perform dimensionality reduction to obtain the fusion feature vector of the simulated acoustic signal :

[0101] ;

[0102] in, represents the constructed weight matrix, express The transposed matrix of .

[0103] Optionally, in step S32, the weight matrix The construction method includes:

[0104] Constructing the weight matrix The objective function is:

[0105] ;

[0106] in, Represents the weight matrix The objective function is represents the target inter-class scatter matrix, Represents the intra-class scatter matrix of targets of the same category;

[0107] The objective function The solution of is transformed into an eigenvalue solution:

[0108] ;

[0109] in, represents the feature vector, represents the eigenvalue;

[0110] Before The feature vectors form a weight matrix .

[0111] S4: Input the fused feature vector of the simulated sound signal into the multi-layer perceptron for training to obtain a trained multi-layer perceptron model:

[0112] S41: Input the fused feature vector of the simulated sound signal into the multi-layer perceptron to obtain the output mapping:

[0113] ;

[0114] in, represents the output mapping, represents the multi-layer perceptron model parameters, Indicates that based on parameters The constructed multi-layer perceptron model function;

[0115] S42: Based on the output mapping and the simulated target signal obtained in step S1, define the loss function of the multilayer perceptron:

[0116] ;

[0117] in, Represents the loss function of the multi-layer perceptron;

[0118] S43: Using the back propagation algorithm to calculate the loss function of the multilayer perceptron Multilayer Perceptron model parameters The gradient of the multilayer perceptron model is updated using the stochastic gradient descent method ;

[0119] S44: Repeat steps S41, S42 and S43 until the loss function Convergence; get the trained multi-layer perceptron model.

[0120] S5: Perform transfer learning on the trained multi-layer perceptron model to achieve denoising of measured sound signals:

[0121] S51: Extraction of measured sound signals The fused feature vector of dimension ;

[0122] S511: Normalize the measured sound signal:

[0123] ;

[0124] in, represents the normalized measured sound signal, Indicates the measured sound signal The minimum value of Indicates the measured sound signal The maximum value of

[0125] S512: The normalized measured sound signal Input long short-term memory network, get The time domain characteristics of ;

[0126] S513: The normalized measured sound signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured sound signal ;

[0127] The frequency domain signal of the measured sound signal Input convolutional neural network and get Frequency domain characteristics ;

[0128] S514: Time domain features Frequency domain characteristics Splice to obtain the fusion feature vector of the measured sound signal :

[0129] .

[0130] S52: The fusion feature vector of the measured sound signal Input the trained multi-layer perceptron model, fine-tune the model parameters, and the model output is the denoised measured sound signal.

[0131] In the embodiment of the present invention, step S1 uses simulation software to obtain a simulated target signal Step S31 uses the vector product method to obtain the fusion matrix of the simulated sound signal It is the expression of the simulated target signal in different types of measured noise signal space; for example, after extracting feature vectors for 10 simulated target signals and 5 measured noise signals respectively, a vector product is performed to obtain a scenario in which 10 simulated target signals are transmitted in 5 measured noise signals, that is, a fusion matrix of 50 simulated sound signals; this can effectively enrich the number of samples of the simulated sound signal when the measured target signal cannot be obtained; step S4 uses the simulated sound signal to obtain a multi-layer perceptron model; step S5 migrates the multi-layer perceptron model to the measured sound signal scenario to achieve denoising.

[0132] Embodiment 2: The present invention also provides an acoustic signal denoising system based on feature fusion, comprising the following five modules:

[0133] Signal acquisition building module: sensor acquisition signal, acquisition of measured noise signal; simulation of speech signal, construction of simulated target signal;

[0134] Feature extraction module: signal normalization, extraction of time domain features, extraction of frequency domain features, and concatenation of features to obtain feature vectors;

[0135] Feature fusion module: expands the dimension of feature vectors, performs vector product operations, and fuses matrix dimensionality reduction;

[0136] Multilayer perceptron training module: multilayer perceptron output mapping, definition of multilayer perceptron loss function, and update of multilayer perceptron model parameters;

[0137] Measured sound signal denoising module: extracts the fusion feature vector of the measured sound signal and fine-tunes the multi-layer perceptron model parameters.

[0138] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0140] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for reducing noise of acoustic signals based on feature fusion, characterized in that: The method comprises: S1: Collect the measured noise signal and construct the simulated target signal, including: The audio sensor is used to collect the acoustic signal in the case of no target, and the acoustic signal in the case of no target is marked as the measured noise signal. ; Use simulation software to construct a speech signal and mark the constructed speech signal as a simulated target signal ; S2: constructing feature vectors of the measured noise signal and the simulated target signal based on the features of the measured noise signal and the simulated target signal, respectively, including: S21: Normalize the measured noise signal and the simulated target signal respectively: ； ； in, represents the normalized measured noise signal, represents the normalized simulated target signal, Represents the measured noise signal The minimum value of Represents the measured noise signal The maximum value of Represents the analog target signal The minimum value of Represents the analog target signal The maximum value of S22: The normalized measured noise signal and the normalized simulated target signal Input into the long short-term memory network respectively, and get The time domain characteristics of and The time domain characteristics of ; S23: The normalized measured noise signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured noise signal ; The normalized simulated target signal Perform short-time Fourier transform to obtain the frequency domain signal of the simulated target signal ; The frequency domain signal of the measured noise signal is and the frequency domain signal of the simulated target signal Input convolutional neural network and get Frequency domain characteristics and Frequency domain characteristics ; S24: The time domain characteristics of the measured noise signal Frequency domain characteristics Splice and get the characteristic vector of the measured noise signal : ； The time domain characteristics of the simulated target signal Frequency domain characteristics Splice to get the characteristic vector of the simulated target signal : ； S3: constructing a fusion feature vector of the simulated acoustic signal based on the feature vectors of the measured noise signal and the simulated target signal; S4: inputting the fused feature vector of the simulated sound signal into the multi-layer perceptron for training to obtain a trained multi-layer perceptron model; S5: Perform transfer learning on the trained multi-layer perceptron model to achieve denoising of measured sound signals.

2. The method for acoustic signal denoising based on feature fusion according to claim 1, characterized in that: The step S3 comprises: S31: and Expand one dimension respectively, and then add the expanded dimensions and Perform vector product operation to obtain the fusion matrix of the simulated sound signal , ,in, express and Original dimension, express OK A real matrix of columns, Representation Matrix belong OK A real matrix of columns; S32: Fusion matrix for analog acoustic signals Perform dimensionality reduction to obtain the fusion feature vector of the simulated acoustic signal : ； in, represents the constructed weight matrix, express The transposed matrix of .

3. The method for reducing noise of acoustic signals based on feature fusion according to claim 2, characterized in that: The step S4 comprises: S41: Input the fused feature vector of the simulated sound signal into the multilayer perceptron to obtain the output mapping: ； in, represents the output mapping, represents the multi-layer perceptron model parameters, Indicates that based on parameters The constructed multi-layer perceptron model function; S42: Based on the output mapping and the simulated target signal obtained in step S1, define the loss function of the multilayer perceptron: ； in, Represents the loss function of the multi-layer perceptron; S43: Using the back propagation algorithm to calculate the loss function of the multilayer perceptron Multilayer Perceptron model parameters The gradient of the multilayer perceptron model is updated using the stochastic gradient descent method ; S44: Repeat steps S41, S42 and S43 until the loss function Convergence; get the trained multi-layer perceptron model.

4. The method for reducing noise of acoustic signals based on feature fusion according to claim 3, characterized in that: The step S5 comprises: S51: Extraction of measured sound signals The fused feature vector of dimension ; S52: The fusion feature vector of the measured sound signal Input the trained multi-layer perceptron model, fine-tune the model parameters, and the model output is the denoised measured sound signal.

5. The method for reducing noise of acoustic signals based on feature fusion according to claim 4, characterized in that: The step S51 comprises: S511: Normalize the measured sound signal: ； in, represents the normalized measured sound signal, Indicates the measured sound signal The minimum value of Indicates the measured sound signal The maximum value of S512: The normalized measured sound signal Input long short-term memory network, get The time domain characteristics of ; S513: The normalized measured sound signal Perform short-time Fourier transform to obtain the frequency domain signal of the measured sound signal ; The frequency domain signal of the measured sound signal Input convolutional neural network and get Frequency domain characteristics ; S514: Time domain features Frequency domain characteristics Splice to obtain the fusion feature vector of the measured sound signal : 。 6. An acoustic signal denoising system based on feature fusion, characterized in that: include: Signal acquisition building block: sensor acquisition signals, acquisition of measured noise signals; Simulate speech signals and construct simulated target signals; Feature extraction module: signal normalization, extraction of time domain features, extraction of frequency domain features, and concatenation of features to obtain feature vectors; Feature fusion module: expands the dimension of feature vectors, performs vector product operations, and fuses matrix dimensionality reduction; Multilayer perceptron training module: multilayer perceptron output mapping, definition of multilayer perceptron loss function, and update of multilayer perceptron model parameters; Measured sound signal denoising module: extracts the fusion feature vector of the measured sound signal and fine-tunes the parameters of the multi-layer perceptron model; To realize the acoustic signal denoising method based on feature fusion as described in any one of claims 1-5.

Images