Sound noise reduction method based on smart phone
This smartphone noise reduction method, which combines timbre recognition and deep neural networks with a multi-microphone array, solves the problem of difficult non-stationary noise processing in existing technologies. It achieves accurate suppression of complex noise and efficient preservation of speech, thereby improving the quality of voice calls and audio recordings.
Patent Information
- Application Number
- CN202511145189.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-15
- Publication Date
- 2025-11-14
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing smartphone noise reduction methods struggle to effectively handle non-stationary noise, especially in multi-person conversation scenarios where they are prone to misinterpreting target speech as noise. Furthermore, they lack personalized recognition capabilities, impacting voice call quality and audio recording performance.
By extracting user timbre features through timbre recognition technology, and combining a deep neural network analysis model and a multi-microphone array, noise is classified and suppressed in real time. The Wiener filtering method is used for post-processing to form a virtual beam pointing to the target sound source and suppress residual noise.
It significantly improves the ability to suppress non-stationary noise, preserves the naturalness of speech, accurately distinguishes effective speech from noise, adapts to complex scenarios and reduces computational complexity, and provides efficient and real-time noise reduction.
Smart Images
Figure CN120954367A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of voice noise reduction technology, specifically a voice noise reduction method based on smartphones. Background Technology
[0002] As smartphones increasingly function as communication tools, entertainment devices, and personal assistants, voice interaction and audio recording have become core application scenarios. However, in actual use, the environment in which smartphones operate is often complex and varied, filled with various background noises. Such noise can seriously affect the quality of voice calls, the accuracy of voice recognition, and the quality of audio recording, especially outdoors, in public places, or in noisy environments, where noise interference is particularly prominent.
[0003] Traditional noise reduction methods, such as spectral subtraction and linear filters, are usually based on the assumption of noise stationarity and are difficult to effectively handle non-stationary noise, such as sudden human voice interference or dynamically changing traffic noise. This can easily lead to distortion or noise residue in the user's target speech, affecting speech intelligibility and naturalness. Furthermore, existing smartphone noise reduction methods are mostly based on noise suppression based on sound source direction or signal strength, lacking the ability to personally identify user timbre characteristics. This may result in the target user's speech being misjudged as noise and suppressed in multi-person conversation scenarios, or other non-target sounds being incorrectly preserved. Summary of the Invention
[0004] (a) Technical problems to be solved
[0005] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a sound noise reduction method based on smartphones, which can effectively solve the problems of the existing technology.
[0006] (II) Technical Solution
[0007] To achieve the above objectives, the present invention provides the following technical solution:
[0008] This invention discloses a sound noise reduction method based on a smartphone, comprising the following steps:
[0009] Step 1: Control permissions through the smartphone, use its built-in microphone to collect user voice samples for timbre recognition, extract the user's timbre features as the unique basis for identifying the target sound source, and store the timbre features in the device's local database as the sound source authentication when inputting voice.
[0010] Step 2: Construct an analysis model using deep neural algorithms, and pre-train the analysis model to learn the mapping relationship from noisy speech signals to clean speech signals;
[0011] Step 3: During the voice input process, the audio signal is collected in real time. Based on the timbre features stored in Step 1, the sounds in the input audio are classified. Sounds that match the timbre features of the target sound source are determined as valid speech, and other sounds that do not match the timbre features are determined as noise and marked.
[0012] Step 4: Using the multi-microphone array equipped on the smartphone, a virtual beam pointing towards the target sound source is formed by adjusting the weighting and phase of the signals from each microphone, initially suppressing the marker noise from non-target directions, and outputting the single-channel signal after beamforming.
[0013] Step 5: Apply the Wiener filtering method to the single-channel signal output in Step 4, design the optimal filter based on the statistical characteristics of the signal and noise, suppress residual noise, and output the enhanced target speech signal.
[0014] Furthermore, the timbre recognition process in step 1 extracts the user's timbre features, performs time-frequency analysis on the user's voice samples, obtains the spectral distribution, fundamental frequency characteristics and harmonic structure features of the timbre, classifies the obtained features, and supports users to update or re-enter timbre samples in different usage scenarios.
[0015] Furthermore, the analytical model in step 2 is constructed using the following steps:
[0016] Step 21: Collect noisy speech datasets including several types of noise and corresponding clean speech datasets, preprocess the audio signals, convert the time-domain signals into time-frequency domain representations using short-time Fourier transform, and extract the time-spectrum as the input features of the model;
[0017] Step 22: Construct an analysis model based on a recurrent neural network. The model includes an input layer, several hidden layers, and an output layer. The hidden layers use a long short-term memory network. The input layer receives the spectrogram features during the input process. The output layer generates the time-frequency masking matrix of the target speech.
[0018] Step 23: Design a loss function to measure the difference between the model output and the real clean speech signal;
[0019] Step 24: Process the input noisy speech time-spectrum using the trained analysis model. Noise suppression is achieved by generating an ideal ratio mask or an ideal binary mask. The masking operation is based on the signal-to-noise ratio estimation of the time-frequency unit. The time-frequency units of the target speech are preserved, while the time-frequency units of non-target speech or noise are suppressed. The enhanced time-spectrum is converted back to the time domain signal through inverse short-time Fourier transform, and clean speech is output.
[0020] Furthermore, the noise types in step 21 include traffic noise, human voice interference, wind noise, and mechanical noise.
[0021] Furthermore, in step 21, the preprocessed dataset is used to train the analysis model through supervised learning. The backpropagation algorithm and gradient descent optimization method are used to update the model parameters. During the training process, batch processing, noise superposition, and volume adjustment are performed.
[0022] Furthermore, during the noise labeling process in step 3, when the audio signal is acquired in real time, the accelerometer data and gyroscope data of the smartphone are used to help determine the orientation and motion state of the smartphone. According to the preset mode, it is determined whether the user is speaking, and the direction of the sound source is estimated. Sounds from non-target directions are further identified as noise, and the noise is classified and labeled by timestamp and spectral features.
[0023] Furthermore, the multi-microphone array in step 4 includes at least two microphones distributed at preset positions on the smartphone, and the beamforming method includes fixed beamforming and adaptive beamforming, wherein the preset target direction of fixed beamforming is the front of the smartphone, while adaptive beamforming dynamically adjusts the beam direction by tracking the target sound source direction in real time.
[0024] Furthermore, the Wiener filtering method in step 5 optimizes the filter parameters by estimating the power spectral density of the signal and noise in real time and using the minimum mean square error criterion.
[0025] (III) Beneficial Effects
[0026] Compared with the known prior art, the technical solution provided by this invention has the following beneficial effects:
[0027] 1. By combining deep neural network learning to learn the mapping relationship from noisy speech to clean speech, the ability to suppress non-stationary and complex noise is significantly improved, while preserving the naturalness of the speech. Furthermore, through the timbre recognition step, the user's timbre is used as the sole basis for identifying the target sound source. Combined with noise labeling technology, effective speech and background noise are accurately distinguished, thereby effectively protecting the target speech and improving the targeting and accuracy of noise reduction.
[0028] 2. By training and quantifying the analysis model and optimizing the design of smartphone hardware units, the computational complexity is reduced, ensuring the real-time performance of the noise reduction process and adapting to the resource limitations of smartphones. Through beamforming technology and combined with the spatial characteristics of multi-microphone arrays, the target sound source direction is focused, and noise from non-target directions is initially suppressed. On this basis, post-processing noise reduction is performed, further improving the noise reduction effect and making up for the shortcomings of existing technologies in the utilization of spatial information. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0030] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation
[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0032] The present invention will be further described below with reference to embodiments.
[0033] This embodiment provides a method for sound noise reduction based on a smartphone, such as... Figure 1 As shown, it includes the following steps:
[0034] Step 1: Control permissions via smartphone, use its built-in microphone to collect user voice samples for timbre recognition, extract the user's timbre features as the unique basis for identifying the target sound source, and store these timbre features in the device's local database for sound source authentication during voice input; The timbre recognition process involves extracting the user's timbre features, performing time-frequency analysis on the user's voice samples to obtain the spectral distribution, fundamental frequency characteristics, and harmonic structure features of the timbre, and classifying the obtained features. It also supports users updating or re-entering timbre samples in different usage scenarios to adapt to changes in voice, ensuring the uniqueness of timbre features and recognition accuracy. The storage of timbre features is encrypted to protect user privacy;
[0035] Step 2: Construct an analysis model using deep neural algorithms. The pre-trained analysis model learns the mapping relationship from noisy speech signals to clean speech signals. The noisy speech signals include the target speech source and background noise. The mapping relationship is used to enhance the target speech through time-frequency domain feature extraction and masking operations.
[0036] Step 3: During voice input, audio signals are acquired in real time. Based on the timbre features stored in Step 1, the sounds in the input audio are classified. Sounds that match the timbre features of the target sound source are determined as valid speech, while other sounds that do not match the timbre features are determined as noise and marked. During the noise marking process, while the audio signals are acquired in real time, the accelerometer and gyroscope data of the smartphone are used to help determine the orientation and motion state of the smartphone. According to the preset mode, it is determined whether the user is speaking, and the direction of the sound source is estimated. Sounds from non-target directions are further confirmed as noise, and the noise is classified and marked by timestamps and spectral features so that subsequent noise reduction steps can adopt differentiated processing strategies for different types of noise.
[0037] Step 4: Utilizing the multi-microphone array equipped on the smartphone, a virtual beam pointing towards the target sound source is formed by adjusting the weighting and phase of the signals from each microphone. This initially suppresses marker noise from non-target directions and outputs a single-channel signal after beamforming. The multi-microphone array includes at least two microphones distributed at preset positions on the smartphone. The beamforming method includes fixed beamforming and adaptive beamforming. Fixed beamforming sets the target direction to the front of the smartphone, while adaptive beamforming dynamically adjusts the beam direction by tracking the target sound source direction in real time. The beamforming process combines the geometric calibration data of the microphone array and the characteristics of the ambient sound field to improve the focusing accuracy of the beam on the target sound source and reduces the leakage of noise from non-target directions through sidelobe suppression technology.
[0038] Step 5: Apply the Wiener filtering method to the single-channel signal output in Step 4. Design an optimal filter based on the statistical characteristics of the signal and noise to suppress residual noise and output the enhanced target speech signal. The Wiener filtering method effectively suppresses residual noise by estimating the power spectral density of the signal and noise in real time and optimizing the filter parameters through the minimum mean square error criterion. In addition, the post-processing noise reduction step can also be combined with spectral subtraction and adaptive filtering as auxiliary noise reduction methods. Spectral subtraction is used to process stationary noise, and adaptive filtering is used to process directional noise with a reference signal, thereby further improving the clarity and naturalness of the output speech signal.
[0039] Compared with existing technologies, this embodiment accurately distinguishes the target user's voice from background noise through timbre recognition technology, avoids false suppression of effective voice, improves the targeting of noise reduction, combines deep neural network learning of the mapping relationship from noisy to clean voice to effectively deal with complex non-stationary noise, and utilizes multi-microphone array beamforming technology to make full use of spatial information to initially suppress non-target direction noise. Through Wiener filtering and auxiliary noise reduction methods, the speech clarity is further improved.
[0040] It can adapt to the resource limitations of smartphones, protect privacy by encrypting and storing voice features locally, and supports real-time processing. It overcomes the shortcomings of existing technologies in terms of adaptability to complex scenarios, personalized protection and real-time performance, and significantly improves voice quality and user experience.
[0041] At other levels, this embodiment also provides an analysis model, the construction process of which includes the following steps:
[0042] Step 21: Collect noisy speech datasets and corresponding clean speech datasets including several noise types, preprocess the audio signals, convert the time-domain signals to time-frequency domain representations using short-time Fourier transform, and extract the time-spectrum as the model input features; the noise types include traffic noise, human voice interference, wind noise, and mechanical noise;
[0043] Step 22: Construct an analysis model based on a recurrent neural network. The model includes an input layer, several hidden layers, and an output layer. The hidden layers use a long short-term memory network to capture the time-series dependence of the audio signal. The input layer receives the spectrogram features, and the output layer generates the time-frequency masking matrix of the target speech.
[0044] Step 23: Design a loss function to measure the difference between the model output and the real clean speech signal. The loss function is a mean squared error loss or a composite loss function combined with perceptual loss to optimize speech quality and intelligibility. Supervised learning training is performed on the analysis model using the preprocessed dataset. The backpropagation algorithm and gradient descent optimization method are used to update the model parameters. During the training process, batch processing, noise superposition, and volume adjustment are performed to improve the robustness of the model to different noise environments and speech intensities.
[0045] Step 24: Process the input noisy speech time-spectrum using the trained analysis model. Noise suppression is achieved by generating an ideal ratio mask or an ideal binary mask. The masking operation is based on the signal-to-noise ratio estimation of the time-frequency unit. The time-frequency units of the target speech are preserved, while the time-frequency units of non-target speech or noise are suppressed. The enhanced time-spectrum is converted back to the time domain signal through inverse short-time Fourier transform, and clean speech is output.
[0046] Compared with existing technologies, the recurrent neural network effectively captures the time-series dependence of audio signals, improves the ability to process dynamic noise, achieves accurate noise suppression through time-spectrum features and masking operations, outperforms traditional signal processing methods, optimizes speech quality and intelligibility using a composite loss function, and enhances the robustness of the model by training with diverse noise datasets.
[0047] In summary, the method of the present invention is applicable to a variety of application scenarios, including but not limited to voice calls, video recording, voice memo recording, live streaming, and speech recognition. It can be implemented at the operating system layer or the application layer. The operating system layer integrates noise reduction functionality through audio codec firmware, while the application layer provides noise reduction services through third-party software development kits or application programming interfaces. It also supports a hybrid cloud and local processing mode, where cloud processing is used for complex data inference to improve noise reduction performance, and local processing is used to protect user privacy and reduce network latency. The two modes can be dynamically switched according to network conditions and device performance.
[0048] This invention uses timbre recognition technology to accurately locate the user's voice as the sole target sound source, effectively distinguishing the target voice from background noise and improving the targeting of noise reduction. Combined with deep neural network learning of the mapping relationship from noisy to clean voice, it can handle complex and non-stationary noise, significantly improving voice clarity and naturalness. Utilizing beamforming technology of a multi-microphone array, it focuses on the target direction to initially suppress noise, and combines Wiener filtering to further optimize the signal, resulting in better noise reduction. It is efficient, practical, and widely applicable, providing smartphone users with a high-quality voice experience.
[0049] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for sound noise reduction based on smartphones, characterized in that, Includes the following steps: Step 1: Control permissions through the smartphone, use its built-in microphone to collect user voice samples for timbre recognition, extract the user's timbre features as the unique basis for identifying the target sound source, and store the timbre features in the device's local database as the sound source authentication when inputting voice. Step 2: Construct an analysis model using deep neural algorithms, and pre-train the analysis model to learn the mapping relationship from noisy speech signals to clean speech signals; Step 3: During the voice input process, the audio signal is collected in real time. Based on the timbre features stored in Step 1, the sounds in the input audio are classified. Sounds that match the timbre features of the target sound source are determined as valid speech, and other sounds that do not match the timbre features are determined as noise and marked. Step 4: Using the multi-microphone array equipped on the smartphone, a virtual beam pointing towards the target sound source is formed by adjusting the weighting and phase of the signals from each microphone, initially suppressing the marker noise from non-target directions, and outputting the single-channel signal after beamforming. Step 5: Apply the Wiener filtering method to the single-channel signal output in Step 4, design the optimal filter based on the statistical characteristics of the signal and noise, suppress residual noise, and output the enhanced target speech signal.
2. The method for sound noise reduction based on a smartphone according to claim 1, characterized in that, The timbre recognition process in step 1 extracts the user's timbre features, performs time-frequency analysis on the user's voice samples, obtains the spectral distribution, fundamental frequency characteristics and harmonic structure features of the timbre, classifies the obtained features, and supports users to update or re-enter timbre samples in different usage scenarios.
3. The method for sound noise reduction based on a smartphone according to claim 1, characterized in that, The analytical model in step 2 is constructed by the following steps: Step 21: Collect noisy speech datasets including several types of noise and corresponding clean speech datasets, preprocess the audio signals, convert the time-domain signals into time-frequency domain representations using short-time Fourier transform, and extract the time-spectrum as the input features of the model; Step 22: Construct an analysis model based on a recurrent neural network. The model includes an input layer, several hidden layers, and an output layer. The hidden layers use a long short-term memory network. The input layer receives the spectrogram features during the input process. The output layer generates the time-frequency masking matrix of the target speech. Step 23: Design a loss function to measure the difference between the model output and the real clean speech signal; Step 24: Process the input noisy speech time-spectrum using the trained analysis model. Noise suppression is achieved by generating an ideal ratio mask or an ideal binary mask. The masking operation is based on the signal-to-noise ratio estimation of the time-frequency unit. The time-frequency units of the target speech are preserved, while the time-frequency units of non-target speech or noise are suppressed. The enhanced time-spectrum is converted back to the time domain signal through inverse short-time Fourier transform, and clean speech is output.
4. The method for sound noise reduction based on a smartphone according to claim 3, characterized in that, The noise types in step 21 include traffic noise, human voice interference, wind noise, and mechanical noise.
5. A method for reducing audio noise based on a smartphone according to claim 3, characterized in that, In step 21, the preprocessed dataset is used to train the analysis model through supervised learning. The backpropagation algorithm and gradient descent optimization method are used to update the model parameters. During the training process, batch processing, noise superposition, and volume adjustment are performed.
6. The method for sound noise reduction based on a smartphone according to claim 1, characterized in that, In step 3, during the process of marking noise, when the audio signal is collected in real time, the accelerometer data and gyroscope data of the smartphone are used to help determine the orientation and motion state of the smartphone. According to the preset mode, it is determined whether the user is speaking, and the direction of the sound source is estimated. Sounds from non-target directions are further identified as noise, and the noise is classified and marked by timestamps and spectral features.
7. The method for sound noise reduction based on a smartphone according to claim 1, characterized in that, The multi-microphone array in step 4 includes at least two microphones distributed at preset positions on the smartphone. The beamforming method includes fixed beamforming and adaptive beamforming. The fixed beamforming presets the target direction as the front of the smartphone, while the adaptive beamforming dynamically adjusts the beam direction by tracking the target sound source direction in real time.
8. A method for reducing audio noise based on a smartphone according to claim 1, characterized in that, The Wiener filtering method in step 5 optimizes the filter parameters by estimating the power spectral density of the signal and noise in real time and using the minimum mean square error criterion.