Sound signal processing method and device and storage medium

Abstract

The invention relates to a sound signal processing method and device and a storage medium. The sound signal processing method comprises the following steps: acquiring an original sound signal, and carrying out wind noise detection on the original sound signal to obtain a frequency boundary value; a first signal and a second signal are determined based on the frequency boundary value, the first signal is used for representing the sound signal interfered by the wind noise, the frequency value of the first signal is smaller than the frequency boundary value, the second signal is used for representing the sound signal not interfered by the wind noise, and the frequency value of the second signal is larger than or equal to the frequency boundary value; performing frequency band expansion on the second signal to obtain a first full-frequency-band signal, the first full-frequency-band signal being a full-frequency-band signal corresponding to the second signal; reconstructing a first signal based on the first full-band signal; and obtaining a target sound signal based on the reconstructed first signal and second signal. According to the invention, the sound signal interfered by the wind noise is subjected to frequency band expansion, and the reconstructed sound signal is obtained.

Description

Technical Field

[0001] The present disclosure relates to the fields of voice communication and audio processing, and in particular to a sound signal processing method, device and storage medium. Background Art

[0002] At present, the quality of voice calls is affected by many factors. Due to various noise interferences in the communication process, the limitation of the link layer at the receiving end, the influence of the band-limited filter, and the requirement of limited storage space, the voices at both ends of the call are mostly narrow-band sound signals with limited bandwidth, which makes it difficult to meet the requirements of high-quality calls, and full-bandwidth communication systems cannot be popularized in reality. As a compromise optimization solution, voice band extension technology (BWE) greatly improves the voice quality of band-limited calls and is currently a hot research topic in audio processing technology.

[0003] Bandwidth extension technology aims to improve the voice quality of band-limited audio signals to meet the requirements of high-quality calls. Generally, bandwidth extension uses low frequencies to extend high frequencies.

[0004] Wind noise is the most common type of noise in nature. It has the characteristics of high noise energy and low frequency range. It easily interferes with sound signals, reduces call quality, and is difficult to suppress. Currently, the commonly used methods for removing wind noise include physical suppression methods and wind noise suppression methods based on modern digital signal processing. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a sound signal processing method, device and storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a sound signal processing method is provided, comprising: acquiring an original sound signal, and performing wind noise detection on the original sound signal to obtain a frequency boundary value; determining a first signal and a second signal based on the frequency boundary value, wherein the first signal is used to represent a sound signal interfered with by wind noise, and the frequency value of the first signal is less than the frequency boundary value, and the second signal is used to represent a sound signal not interfered with by wind noise, and the frequency value of the second signal is greater than or equal to the frequency boundary value; performing frequency band expansion on the second signal to obtain a first full-band signal, and the first full-band signal is a full-band signal corresponding to the second signal; reconstructing the first signal based on the first full-band signal; and obtaining a target sound signal based on the reconstructed first signal and the second signal.

[0007] In one embodiment, the frequency band expansion of the second signal to obtain a first full-band signal includes: based on the frequency boundary value, filtering the original sound signal to obtain a first frequency band signal of the second signal, wherein the first frequency band signal is a signal with a frequency greater than or equal to the frequency boundary value and less than or equal to frequency N, and N is an adjustable frequency parameter; frequency band expansion of the first frequency band signal to obtain a second full-band signal, the second full-band signal being a full-band signal corresponding to the first frequency band signal; and using the second full-band signal as the first full-band signal.

[0008] In one embodiment, reconstructing the first signal based on the first full-band signal includes: performing a filtering operation on the first full-band signal to obtain a second frequency band signal, wherein the second frequency band signal is a signal having a frequency greater than or equal to a frequency M and less than or equal to the frequency boundary value, and the frequency M is a signal greater than zero and less than a frequency threshold; using the second frequency band signal as the reconstructed first signal.

[0009] In one embodiment, obtaining the target sound signal based on the reconstructed first signal and the second signal includes: splicing the second signal and the reconstructed first signal to obtain a third signal; performing spectrum envelope extraction on the original sound signal and the third signal; and obtaining the target sound signal based on the spectrum envelope extraction result and the third signal.

[0010] In one embodiment, the extracting the spectrum envelope of the original sound signal and the third signal includes: transforming the original sound signal and the third signal from the time domain to the frequency domain based on a preset signal conversion method to obtain the frequency domain signal of the original sound signal and the frequency domain signal of the third signal; taking the absolute values ​​of the corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal to obtain the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; performing spectrum envelope extraction on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

[0011] In one embodiment, the target sound signal is obtained based on the spectrum envelope extraction result and the third signal, including: obtaining an energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; performing gain calculation on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal and the energy adjustment value to obtain a gain calculation result; performing spectrum envelope conversion based on the gain calculation result and the frequency domain signal of the third signal to obtain a frequency domain signal after spectrum envelope conversion, wherein the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value; converting the frequency domain signal after spectrum envelope conversion into a time domain signal based on a preset signal conversion method to obtain a target sound signal.

[0012] In one embodiment, the energy adjustment value of the second frequency band signal is obtained based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal, including: obtaining the root mean square of the original sound signal spectrum based on the frequency domain amplitude information of the original sound signal; obtaining the root mean square of the third signal spectrum based on the frequency domain amplitude information of the third signal; obtaining the energy adjustment value of the second frequency band signal based on the ratio of the root mean square of the third signal spectrum to the root mean square of the original sound signal spectrum; wherein the frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to a frequency boundary value.

[0013] According to a second aspect of an embodiment of the present disclosure, a sound signal processing device is provided, comprising: an acquisition unit, used to acquire an original sound signal, and perform wind noise detection on the original sound signal to obtain a frequency boundary value; a determination unit, used to determine a first signal and a second signal based on the frequency boundary value, wherein the first signal is used to represent a sound signal interfered with by wind noise, and the frequency value of the first signal is less than the frequency boundary value, and the second signal is used to represent a sound signal not interfered with by wind noise, and the frequency value of the second signal is greater than or equal to the frequency boundary value; an expansion unit, used to perform frequency band expansion on the second signal to obtain a first full-band signal, and the first full-band signal is a full-band signal corresponding to the second signal; a reconstruction unit, used to reconstruct the first signal based on the first full-band signal; and an execution unit, used to obtain a target sound signal based on the reconstructed first signal and the second signal.

[0014] In one embodiment, the expansion unit performs frequency band expansion on the second signal to obtain a first full-band signal in the following manner: based on the frequency boundary value, filtering the original sound signal to obtain a first frequency band signal of the second signal, wherein the first frequency band signal is a signal having a frequency greater than or equal to the frequency boundary value and less than or equal to frequency N, where N is an adjustable frequency parameter; frequency band expansion is performed on the first frequency band signal to obtain a second full-band signal, wherein the second full-band signal is a full-band signal corresponding to the first frequency band signal; and the second full-band signal is used as the first full-band signal.

[0015] In one embodiment, the reconstruction unit reconstructs the first signal based on the first full-band signal in the following manner: filtering the first full-band signal to obtain a second frequency band signal, wherein the second frequency band signal is a signal having a frequency greater than or equal to a frequency M and less than or equal to the frequency boundary value, and the frequency M is a signal greater than zero and less than a frequency threshold; and using the second frequency band signal as the reconstructed first signal.

[0016] In one embodiment, the execution unit obtains the target sound signal based on the reconstructed first signal and the second signal in the following manner: concatenating the second signal and the reconstructed first signal to obtain a third signal; performing spectrum envelope extraction on the original sound signal and the third signal; and obtaining the target sound signal based on the spectrum envelope extraction result and the third signal.

[0017] In one embodiment, the execution unit performs spectrum envelope extraction on the original sound signal and the third signal in the following manner: transforming the original sound signal and the third signal from the time domain to the frequency domain based on a preset signal conversion method to obtain the frequency domain signal of the original sound signal and the frequency domain signal of the third signal; taking the absolute values ​​of the corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal to obtain the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; performing spectrum envelope extraction on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

[0018] In one embodiment, the execution unit obtains the target sound signal based on the spectrum envelope extraction result and the third signal in the following manner: based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal, obtain the energy adjustment value of the second frequency band signal; perform gain calculation on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal and the energy adjustment value to obtain the result of the gain calculation; perform spectrum envelope conversion on the result of the gain calculation and the frequency domain signal of the third signal to obtain the frequency domain signal after the spectrum envelope conversion, wherein the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value; convert the frequency domain signal after the spectrum envelope conversion into a time domain signal based on a preset signal conversion method to obtain the target sound signal.

[0019] In one embodiment, the execution unit obtains the energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal in the following manner: based on the frequency domain amplitude information of the original sound signal, obtain the root mean square of the original sound signal spectrum; based on the frequency domain amplitude information of the third signal, obtain the root mean square of the third signal spectrum; based on the ratio of the root mean square of the third signal spectrum to the root mean square of the original sound signal spectrum, obtain the energy adjustment value of the second frequency band signal; wherein the frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to a frequency boundary value.

[0020] According to a third aspect of an embodiment of the present disclosure, a sound signal processing device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the sound signal processing method described in the first aspect or any one of the embodiments of the first aspect.

[0021] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the sound signal processing method described in the first aspect or any one of the embodiments of the first aspect.

[0022] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: inputting a sound signal into a wind noise detection module to obtain a frequency boundary value of the sound signal contaminated by wind noise, and based on the frequency boundary value, performing frequency band expansion on the sound signal not contaminated by wind noise to obtain an expanded full-band signal, and based on the expanded full-band signal and the sound signal not contaminated by wind noise, obtaining a reconstructed target sound signal contaminated by wind noise.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0025] Figure 1 The figure is a schematic diagram showing a voice call in a wind noise scenario according to an exemplary embodiment.

[0026] Figure 2 The present invention is a flow chart showing the overall principle of removing wind noise by using reverse frequency band expansion according to an exemplary embodiment.

[0027] Figure 3 The figure is a flow chart of a sound signal processing method according to an exemplary embodiment.

[0028] Figure 4 The present invention is a flowchart showing a method of performing frequency band expansion on a second signal to obtain a first full-band signal according to an exemplary embodiment.

[0029] Figure 5 The figure is a principle flow chart of a reverse frequency band extension module according to an exemplary embodiment.

[0030] Figure 6 The present invention is a flowchart showing a method of reconstructing a first signal based on a first full-band signal according to an exemplary embodiment.

[0031] Figure 7 The figure is a principle flow chart of a post-processing module according to an exemplary embodiment.

[0032] Figure 8 The present invention is a flowchart showing a method of obtaining a target sound signal based on a reconstructed first signal and a second signal according to an exemplary embodiment.

[0033] Fig. 9 The invention is a flow chart showing a method of extracting spectrum envelopes from an original sound signal and a third signal according to an exemplary embodiment.

[0034] Fig.10 The present invention is a flowchart showing a method of obtaining a target sound signal based on a spectrum envelope extraction result and a third signal according to an exemplary embodiment.

[0035] Fig.11 The present invention is a flowchart showing a method of obtaining an energy adjustment value of a second frequency band signal based on frequency domain amplitude information of an original sound signal and frequency domain amplitude information of a third signal according to an exemplary embodiment.

[0036] Fig.12The present invention is a block diagram of a sound signal processing device according to an exemplary embodiment.

[0037] Fig.13 The present invention is a block diagram of a sound signal processing device according to an exemplary embodiment.

[0038] Fig.14 The present invention is a block diagram of a sound signal processing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0039] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0040] The basic idea of ​​the present disclosure is to use high frequency to expand low frequency, which is called reverse frequency band expansion. And because the basic idea of ​​the present disclosure is to use reverse frequency band expansion to remove wind noise, the frequency band expansion and wind noise removal related technologies are now explained separately.

[0041] Bandwidth extension (BWE) includes blind band extension and non-blind band extension:

[0042] (1) Non-blind frequency band expansion: Special coding and decoding technologies are used to reconstruct the lost spectrum. Common technologies include Spectral Band Relication (SBR) and Accurate Spectral Replacement (ASR).

[0043] (2) Blind band extension: Spectrum reconstruction is performed without any prior information about the missing spectrum. Common techniques include half-wave rectification (HW), full-wave rectification (FW), phase space reconstruction (PSR), etc.

[0044] Wind noise suppression technology includes wind noise suppression methods for single microphone systems based on digital signal processing: methods based on linear predictive coding (LPC), frequency domain subband methods, non-negative sparse matrix methods, etc., as well as wind noise suppression methods for multi-sensor systems (using the correlation characteristics of multi-channel signals to suppress wind noise), wind noise suppression methods based on neural networks. There are also methods based on physical suppression of wind noise, which are related to hardware design.

[0045] Figure 1is a schematic diagram showing a voice call in a wind noise scenario according to an exemplary embodiment. Figure 1 As shown. In daily voice call scenarios, the remote device A and the near-end device B communicate with each other. The calling devices can be mobile phones, wireless headphones (True Wireless Stereo, TWS), smart watches and other devices. The near-end device B is in an environment with relatively strong wind noise, such as strong airflow and windy weather outdoors, or a high-speed car environment, which has a serious impact and interference on the sound pickup process of the near-end B, reducing the intelligibility of the sound signal and making it difficult to suppress, causing the remote device A to hear the signal B' interfered by the wind noise, making the user's call experience particularly poor. The present disclosure applies the reverse band extension technology to this scenario, and reversely reconstructs the low-frequency signal polluted by wind noise, which can greatly improve the voice quality of the call.

[0046] Common methods of removing wind noise, while suppressing wind noise, also cause certain damage to the sound signal. Especially when the wind speed is high and the wind noise masks the sound signal, these methods seriously reduce the intelligibility of the sound signal.

[0047] Based on this, the present invention applies band extension (BWE) to wind noise suppression technology to expand the sound signal from the high frequency band to the low frequency band, and reconstruct the low frequency signal spectrum contaminated by wind noise. This method can effectively suppress wind noise and improve voice call quality.

[0048] For the convenience of description, the present disclosure may refer to the original sound signal as an input signal (input).

[0049] Figure 2 FIG. 1 is a flow chart showing the overall principle of removing wind noise by using reverse frequency band expansion according to an exemplary embodiment. Figure 2 The embodiment of the present disclosure is divided into three parts:

[0050] The first part is a wind-noise detection module, which performs wind-noise detection on the input signal and provides a frequency boundary value wind_freq interfered by the wind noise. The frequency boundary value divides the input signal into a first signal and a second signal.

[0051] The second part is the band extension (BWE) module. According to the wind noise detection result, the frequency point that is not interfered by wind noise, that is, the first frequency band signal, is band expanded to obtain the expanded full-band signal, and the required low-frequency harmonic part, that is, the second frequency band signal, is obtained. The low-frequency harmonic part does not carry wind noise information, that is, it is not interfered by wind noise.

[0052] The third part is the post-processing module, which splices and integrates the extended low-frequency harmonic part, i.e., the second frequency band signal, and the high-frequency part of the original signal, i.e., the second signal, to obtain the third signal, and performs spectrum envelope conversion on the extended low-frequency speech, i.e., the second frequency band signal, to make its timbre and speech content closer to the original speech, thereby improving the speech intelligibility of the extended signal, and finally obtaining the result after wind noise suppression, i.e., the target sound signal.

[0053] The following is an explanation of the methods applied to the three modules in the embodiments of the present disclosure:

[0054] Figure 3 is a flow chart of a sound signal processing method according to an exemplary embodiment. Figure 3 As shown, the method includes the following steps.

[0055] In step S11, an original sound signal is acquired, and wind noise detection is performed on the original sound signal to obtain a frequency boundary value.

[0056] In the embodiment of the present disclosure, an input signal (input) input to the electronic device, ie, an original sound signal, is obtained, and the signal is input to a wind noise detection module for wind noise detection to obtain a frequency boundary value wind_freq interfered by wind noise.

[0057] In step S12, the first signal and the second signal are determined based on the frequency boundary value.

[0058] In an embodiment of the present disclosure, the first signal is used to represent a sound signal disturbed by wind noise, and the frequency value of the first signal is less than the frequency boundary value. The second signal is used to represent a sound signal not disturbed by wind noise, and the frequency value of the second signal is greater than or equal to the frequency boundary value.

[0059] Exemplarily, a frequency band of 0 to wind_freq KHz represents a frequency band disturbed by wind noise, namely, a first signal, and a frequency band greater than wind_freq KHz represents a speech frequency band not disturbed by wind noise, namely, a second signal.

[0060] In step S13, the second signal is subjected to frequency band extension to obtain a first full-band signal.

[0061] In the embodiment of the present disclosure, the first full-band signal is a full-band signal corresponding to the second signal.

[0062] In the embodiment of the present disclosure, the input signal is divided into a first signal and a second signal based on a frequency boundary value, and the second signal is band-extended to obtain a first full-band signal that is not contaminated by wind noise.

[0063] In step S14, the first signal is reconstructed based on the first full-band signal.

[0064] In step S15, a target sound signal is obtained based on the reconstructed first signal and the second signal.

[0065] In the disclosed embodiment, the reconstructed first signal and the second signal are spliced, and the spliced ​​signal and the original sound signal are post-processed to obtain a target sound signal.

[0066] In the embodiment of the present disclosure, a frequency boundary value interfered by wind noise is obtained through a wind noise detection module, and the input signal is divided into a first signal interfered by wind noise and a second signal not interfered by wind noise based on the input signal and the frequency boundary value, and the frequency band of the second signal not interfered by wind noise is band expanded to obtain a first full-band signal, and based on the first full-band signal, the first signal is reconstructed, the reconstructed first signal and the second signal are spliced, and the spliced ​​signal is post-processed with the original sound signal to obtain a target sound signal, that is, the result of suppressing wind noise in the original signal, which has a good effect of eliminating wind noise.

[0067] Figure 4 FIG. 1 is a flow chart showing a method of performing frequency band expansion on a second signal to obtain a first full-band signal according to an exemplary embodiment. Figure 4 As shown, the following steps are included.

[0068] In step S21, based on the frequency boundary value, a filtering operation is performed on the original sound signal to obtain a first frequency band signal of the second signal.

[0069] In the embodiment of the present disclosure, the first frequency band signal is a signal whose frequency is greater than or equal to the frequency boundary value and less than or equal to the frequency N, where N is an adjustable frequency parameter.

[0070] Figure 5 is a flow chart showing the principle of a reverse frequency band extension module according to an exemplary embodiment. Figure 5 In the disclosed embodiment, blind band extension is adopted, that is, spectrum reconstruction is performed without any prior information about the missing spectrum. There are many spectrum reconstruction methods, and full-wave rectification technology is used as an example for explanation.

[0071] In the disclosed embodiment, the input signal is passed through a bandpass filter, and the original speech band to be expanded, i.e., the first frequency band signal, is selected. We set the starting frequency and cutoff frequency of the bandpass filter according to the frequency boundary value wind_freq given by the wind noise detection module. An effective signal of N KHz is selected as the original speech band, i.e., the first frequency band signal, that is, the starting frequency of the bandpass filter is wind_freq KHz, and the cutoff frequency of the bandpass filter is (wind_freq+N) KHz, where N is an adjustable frequency parameter, which can be corrected in real time according to the frequency band of wind noise interference. fil_out1 is a bandpass signal whose output result of the bandpass filter is wind_freq~(wind_freq+N KHz), i.e., the first frequency band signal.

[0072] In step S22, the first frequency band signal is frequency-extended to obtain a second full-band signal.

[0073] In the implementation of the present disclosure, the second full-band signal is a full-band signal corresponding to the first frequency band signal.

[0074] In the disclosed embodiment, the frequency band extension unit is called a nonlinear device (NLD), and the disclosed embodiment is implemented by full-wave rectification, that is, taking the absolute value abs of the input first frequency band signal fil_out1, extending the signal to the full frequency band, and obtaining accurate information of each harmonic. The output result of the NLD frequency band extension is the second full-band signal (nld_out) after the first frequency band signal is extended.

[0075] In step S23, the second full-band signal is used as the first full-band signal.

[0076] In the disclosed embodiment, the first frequency band signal with a frequency of (wind_freq+N)KHz is frequency-band expanded to obtain a second full-band signal, the second signal is frequency-band expanded to obtain a first full-band signal, the second full-band signal is used as the first full-band signal, and subsequent operations are performed based on the second full-band signal. In the disclosed embodiment, based on reverse frequency band expansion, the sound signal interfered by wind noise is frequency-band expanded, thereby improving the voice quality of calls with limited frequency bands and providing a good user experience.

[0077] Figure 6 is a flowchart showing a method of reconstructing a first signal based on a first full-band signal according to an exemplary embodiment. Figure 6 As shown, the following steps are included.

[0078] In step S31, a filtering operation is performed on the first full-band signal to obtain a second frequency band signal.

[0079] In the embodiment of the present disclosure, the second full-band signal after the first frequency band signal is expanded is passed through a bandpass filter, and the required second frequency band signal of the expanded frequency band is selected. The present disclosure is mainly for suppressing the wind noise in the low frequency band, that is, the signal with a frequency less than the frequency boundary value, and does not contain a DC component, so the starting frequency of the bandpass filter is set to M KHz, and the cutoff frequency is wind_freq KHz, that is, M~wind_freq KHz is selected as the expanded frequency band. The frequency M can be set to a small value greater than zero, such as 100Hz, to filter out the DC component. The output result of the bandpass filter is fil_out2, which represents the second frequency band signal with a frequency of M~wind_freq KHz.

[0080] In the embodiment of the present disclosure, the second frequency band signal is a signal whose frequency is greater than or equal to frequency M and less than or equal to a frequency boundary value, and frequency M is a signal greater than zero and less than a frequency threshold.

[0081] In step S32, the second frequency band signal is used as the reconstructed first signal.

[0082] Figure 7 is a flow chart showing the principle of a post-processing module according to an exemplary embodiment. Figure 7 In the disclosed embodiment, the main function of the frequency band extension post-processing is to splice and integrate the extended low-frequency harmonic part (M~wind_freq KHz), i.e., the second frequency band signal, and the high-frequency part (greater than wind_freq KHz), i.e., the second signal, of the original sound signal to obtain a third signal, and obtain a target sound signal based on the third signal and the original sound signal, i.e., the result of suppressing wind noise in the original signal.

[0083] In the disclosed embodiment, the original sound signal is expanded from a high frequency band to a low frequency band, the signal interfered by wind noise is reconstructed, and the target sound signal is obtained, thereby effectively suppressing the interference of wind noise.

[0084] Figure 8 is a flow chart showing a method of obtaining a target sound signal based on a reconstructed first signal and a second signal according to an exemplary embodiment. Figure 8 As shown, the following steps are included.

[0085] In step S41, the second signal and the reconstructed first signal are concatenated to obtain a third signal.

[0086] In the disclosed embodiment, the original sound signal input is high-pass filtered, and the cutoff frequency of the high-pass filter is set according to the frequency boundary value wind_freq of the wind noise detection, and a high-pass signal with a cutoff frequency of wind_freq KHz is obtained. The output result of the high-pass filtering is fil_out3, which indicates a high-pass signal greater than wind_freq KHz, i.e., the second signal. The high-pass signal (greater than wind_freq KHz), i.e., the second signal, is spliced ​​with the second frequency band signal (fil_out2) with a frequency of M to wind_freq KHz output by the frequency band extension BWE, i.e., fil_out3 and fil_out2 signals are spliced ​​to obtain an extended full-band signal, represented by add_out, i.e., the third signal.

[0087] In step S42, spectrum envelope extraction is performed on the original sound signal and the third signal.

[0088] In the embodiment of the present disclosure, the spectrum envelope of the original sound signal and the spectrum envelope of the third signal are obtained.

[0089] In step S43, a target sound signal is obtained based on the spectrum envelope extraction result and the third signal.

[0090] In the disclosed embodiment, the spectral envelope of the original sound signal, the spectral envelope of the third signal and the energy adjustment value are gain calculated, and the obtained gain calculation result and the frequency domain signal of the third signal are envelope converted, so that the timbre and voice content of the low frequency band are closer to the original voice.

[0091] Fig. 9 is a flow chart showing a method of extracting spectrum envelopes from an original sound signal and a third signal according to an exemplary embodiment. Fig. 9 As shown, the following steps are included.

[0092] In step S51, the original sound signal and the third signal are transformed from the time domain to the frequency domain based on a preset signal conversion method to obtain a frequency domain signal of the original sound signal and a frequency domain signal of the third signal.

[0093] In the disclosed embodiment, the original signal input is transformed into the frequency domain, that is, the input is subjected to a short-time Fourier transform (STFT) operation to obtain the frequency domain signal of the original signal, that is, the frequency domain signal of the sound signal, represented by fft_out2. The spliced ​​full-band signal (add_out), that is, the third signal, is transformed from the time domain to the frequency domain, that is, the STFT operation is performed to obtain the frequency domain signal of the third signal, represented by fft_out1. One way of fft_out1 is sent to the abs module to take the absolute value for subsequent spectrum envelope extraction, and the other way is used for spectrum envelope conversion. The two signals are exactly the same.

[0094] In step S52, the absolute values ​​of the corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal are taken to obtain the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal.

[0095] In the disclosed embodiment, the frequency domain signal fft_out2 of the original sound signal and the spliced ​​full-band frequency domain signal, i.e., the third signal fft_out1, are respectively sent to the abs module to obtain the frequency domain amplitude information of the two signals. abs_out1 represents the frequency domain amplitude of the original sound signal, and abs_out2 represents the frequency domain amplitude of the spliced ​​full-band signal, i.e., the third signal. The frequency domain amplitude calculation formula is as follows:

[0096] abs_out(i)=sqrt(real(i)+imag(t))

[0097] Where real(i) represents the real part of the frequency domain, imag(i) represents the imaginary part of the frequency domain, abs_out(i) represents the frequency domain amplitude of the signal, and sqrt is the square root function. abs_out1 and abs_out2 are both in array form.

[0098] In step S53, spectrum envelope extraction is performed on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

[0099] In the embodiment of the present disclosure, the frequency domain amplitude abs_out1 of the original sound signal and the frequency domain amplitude abs_out2 of the full-band signal after splicing, i.e., the third signal, are respectively sent to the spectrum envelope extraction module. There are many methods for spectrum envelope extraction. The present disclosure takes mean filtering as an example, and calculates the mean of up to (2*n+1) frequency points (including the current point) around each frequency point as the envelope value of the current frequency point. env_out1 represents the spectrum envelope of the original sound signal, and env_out2 represents the spectrum envelope of the full-band signal after splicing, i.e., the third signal. The envelope extraction mean filtering formula is as follows, and the filter length value n can be adjusted according to actual conditions:

[0100]

[0101] Where n is the filter length, abs_out is the frequency domain amplitude information, and env_out is the spectrum envelope of the signal in array form.

[0102] Fig.10 is a flow chart showing a method of obtaining a target sound signal based on a spectrum envelope extraction result and a third signal according to an exemplary embodiment. Fig.10 As shown, the following steps are included.

[0103] In step S61, the energy adjustment value of the second frequency band signal is obtained based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal.

[0104] In the disclosed embodiment, the abs absolute value module also needs to calculate the energy adjustment value of the extended low frequency spectrum (M~wind_freqKHz), which is an engineering value and is changed according to the actual situation during debugging.

[0105] In step S62, gain calculation is performed on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal, and the energy adjustment value to obtain a gain calculation result.

[0106] In the disclosed embodiment, the spectrum envelope env_out1 of the original sound signal, the spectrum envelope env_out2 of the spliced ​​full-band signal, i.e., the third signal, and the expanded low-spectrum energy adjustment value amp_gain are sent to the gain calculation module to obtain the gain calculation result:

[0107] env_gain(i)=amp_gain*(env_out1(i) / env_out2(i))

[0108] Among them, env_out1 and env_out2 are related to the frequency and are only valid in the range of M to wind_freq KHz. amp_gain is independent of the frequency and is a fixed value.

[0109] In step S63, spectrum envelope conversion is performed based on the gain calculation result and the frequency domain signal of the third signal to obtain a frequency domain signal after spectrum envelope conversion.

[0110] In the disclosed embodiment, the gain value env_gain of the spectrum envelope conversion and the spectrum fft_out1 of the spliced ​​full-band signal, i.e., the third signal, are sent to the envelope conversion module for spectrum envelope conversion. It is only necessary to convert the extended M~wind_freq KHz low frequency band. In fact, the real part and the imaginary part of fft_out1 are multiplied by the gain value env_gain respectively, and the output is the frequency domain signal env_out after envelope conversion. The envelope conversion formula is as follows:

[0111] env_out(i).real=env_gain(i)*fft_out1(i).real

[0112] env_out(i).imag=env_gain(i)*fft_out1(i).imag

[0113] Among them, fft_out1().real is the real part of fft_out1, fft_out1().image is the imaginary part of fft_out1, env_gain() is the gain value, env_out().real is the real part of the frequency domain signal after envelope conversion, and env_out().imag is the imaginary part of the frequency domain signal after envelope conversion.

[0114] In the disclosed embodiment, in order to obtain the new spectrum of the extended speech, it is necessary to obtain the spectrum envelope of the original speech and the spectrum envelope of the extended speech, and obtain the ratio of the two speech spectrum envelopes, and then reconstruct the new spectrum of the extended speech. This process is called spectrum envelope conversion. The spectrum envelope conversion is performed on the extended low-frequency speech, that is, the second frequency band signal, so that the timbre and speech content of the low-frequency band speech are closer to the original speech, and the speech intelligibility of the extended signal is improved.

[0115] In the embodiment of the present disclosure, the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value.

[0116] In step S64, the frequency domain signal after the spectrum envelope conversion is converted into a time domain signal based on a preset signal conversion method to obtain a target sound signal.

[0117] In the disclosed embodiment, the spectrum envelope conversion signal env_out is subjected to an inverse ISTFT transform to be transformed into a time domain signal as the final output result output, that is, a target sound signal is obtained.

[0118] Fig.11 is a flow chart showing a method of obtaining an energy adjustment value of a second frequency band signal based on frequency domain amplitude information of an original sound signal and frequency domain amplitude information of a third signal according to an exemplary embodiment, such as Fig.11 As shown, the following steps are included.

[0119] In step S71, the root mean square of the spectrum of the original sound signal is obtained based on the frequency domain amplitude information of the original sound signal.

[0120] In step S72, the root mean square of the spectrum of the third signal is obtained based on the frequency domain amplitude information of the third signal.

[0121] In step S73, the energy adjustment value of the second frequency band signal is obtained based on the ratio of the root mean square of the third signal spectrum to the root mean square of the original sound signal spectrum.

[0122] In the disclosed embodiment, the energy adjustment value of the extended low spectrum (M~wind_freqKHz) needs to be calculated in the abs absolute value module. This value is an engineering value and is changed according to the actual situation during debugging. The root mean square rms1 of the M~wind_freq KHz extended signal spectrum is calculated according to abs_out2, and the root mean square rms2 of the M~wind_freq KHz original signal spectrum is calculated according to abs_out1. Then the extended low spectrum energy adjustment value amp_gain = alpha*rms1 / rms2, alpha is the adjustment coefficient. The root mean square rms calculation formula is as follows:

[0123]

[0124] Among them, sqrt is the square root function, abs_out is the amplitude information of the signal, and ∑ is the summation function.

[0125] In the embodiment of the present disclosure, the frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to a frequency boundary value.

[0126] Through the present disclosure, the band extension (BWE) is combined with the wind noise suppression technology, and the reverse band extension is used to remove the wind noise. The low frequency band is expanded by the high frequency band, and the clean low frequency signal of the band extension is reversely replaced with the low frequency signal interfered by the wind noise, that is, the low frequency band frequency point interfered by the wind noise is reconstructed, and the wind noise is effectively removed, thereby improving the call quality. In addition, the present disclosure has a low computational complexity, and most of the wind noise removal operations used are common operations such as taking absolute values, multiplication and division, without complex operations such as adaptive filtering and exponential logarithm, and a good effect is achieved with simple operations and low computational complexity, and it is convenient for engineering applications.

[0127] Based on the same concept, an embodiment of the present disclosure also provides a sound signal processing device.

[0128] It is understandable that the sound signal processing device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0129] Fig.12 FIG. 1 is a block diagram of a sound signal processing device 100 according to an exemplary embodiment. Fig.12 The device 100 includes an acquisition unit 101, a determination unit 102, an expansion unit 103, a reconstruction unit 104 and an execution unit 105.

[0130] The acquisition unit 101 is used to acquire the original sound signal and perform wind noise detection on the original sound signal to obtain a frequency boundary value; the determination unit 102 is used to determine a first signal and a second signal based on the frequency boundary value, wherein the first signal is used to represent a sound signal interfered by wind noise, and the frequency value of the first signal is less than the frequency boundary value, and the second signal is used to represent a sound signal not interfered by wind noise, and the frequency value of the second signal is greater than or equal to the frequency boundary value; the expansion unit 103 is used to perform frequency band expansion on the second signal to obtain a first full-band signal, and the first full-band signal is a full-band signal corresponding to the second signal; the reconstruction unit 104 is used to reconstruct the first signal based on the first full-band signal; the execution unit 105 is used to obtain a target sound signal based on the reconstructed first signal and second signal.

[0131] In one embodiment, the expansion unit 103 performs frequency band expansion on the second signal to obtain a first full-band signal in the following manner: based on the frequency boundary value, the original sound signal is filtered to obtain a first frequency band signal of the second signal, wherein the first frequency band signal is a signal whose frequency is greater than or equal to the frequency boundary value and less than or equal to the frequency N, where N is an adjustable frequency parameter; the first frequency band signal is frequency band expanded to obtain a second full-band signal, the second full-band signal is a full-band signal corresponding to the first frequency band signal; and the second full-band signal is used as the first full-band signal.

[0132] In one embodiment, the reconstruction unit 104 reconstructs the first signal based on the first full-band signal in the following manner: filtering the first full-band signal to obtain a second frequency band signal, wherein the second frequency band signal is a signal whose frequency is greater than or equal to frequency M and less than or equal to a frequency boundary value, and frequency M is a signal greater than zero and less than a frequency threshold; and the second frequency band signal is used as the reconstructed first signal.

[0133] In one embodiment, the execution unit 105 obtains the target sound signal based on the reconstructed first signal and the second signal in the following manner: concatenating the second signal and the reconstructed first signal to obtain a third signal; performing spectrum envelope extraction on the original sound signal and the third signal; and obtaining the target sound signal based on the spectrum envelope extraction result and the third signal.

[0134] In one embodiment, the execution unit 105 performs spectrum envelope extraction on the original sound signal and the third signal in the following manner: transforming the original sound signal and the third signal from the time domain to the frequency domain based on a preset signal conversion method to obtain the frequency domain signal of the original sound signal and the frequency domain signal of the third signal; taking the absolute values ​​of the corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal to obtain the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; performing spectrum envelope extraction on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

[0135] In one embodiment, the execution unit 105 obtains the target sound signal based on the spectrum envelope extraction result and the third signal in the following manner: based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal, an energy adjustment value of the second frequency band signal is obtained; gain calculation is performed on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal, and the energy adjustment value to obtain a result of the gain calculation; spectrum envelope conversion is performed on the result of the gain calculation and the frequency domain signal of the third signal to obtain a frequency domain signal after spectrum envelope conversion, wherein the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value; the frequency domain signal after spectrum envelope conversion is converted into a time domain signal based on a preset signal conversion method to obtain a target sound signal.

[0136] In one embodiment, the execution unit 105 obtains the energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal in the following manner: based on the frequency domain amplitude information of the original sound signal, obtain the root mean square of the original sound signal spectrum; based on the frequency domain amplitude information of the third signal, obtain the root mean square of the third signal spectrum; based on the ratio of the root mean square of the third signal spectrum to the root mean square of the original sound signal spectrum, obtain the energy adjustment value of the second frequency band signal; wherein the frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to the frequency boundary value.

[0137] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0138] Fig.13 2 is a block diagram of a device 200 for sound signal processing according to an exemplary embodiment. The device 200 may be provided as a terminal. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0139] Reference Fig.13 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0140] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0141] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0142] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0143] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0144] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0145] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0146] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0147] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0148] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0149] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0150] Fig.14 is a block diagram of a device 300 for sound signal processing according to an exemplary embodiment. For example, the device 300 may be provided as a server. Fig.14, the apparatus 300 includes a processing component 322, which further includes one or more processors, and a memory resource represented by a memory 332 for storing instructions, such as an application, that can be executed by the processing component 322. The application stored in the memory 332 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 322 is configured to execute the instructions to perform the above method.

[0151] The device 300 may also include a power supply component 326 configured to perform power management of the device 300, a wired or wireless network interface 350 configured to connect the device 300 to a network, and an input / output (I / O) interface 358. The device 300 may operate based on an operating system stored in the memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0152] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 332 including instructions, which can be executed by the processing component 322 of the device 300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0153] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0154] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0155] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0156] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0157] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0158] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0159] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A sound signal processing method, characterized in that: include: Acquire an original sound signal, and perform wind noise detection on the original sound signal to obtain a frequency cutoff value; Determine a first signal and a second signal based on the frequency demarcation value, wherein the first signal is used to represent a sound signal disturbed by wind noise, and the frequency value of the first signal is less than the frequency demarcation value, and the second signal is used to represent a sound signal not disturbed by wind noise, and the frequency value of the second signal is greater than or equal to the frequency demarcation value; Performing frequency band expansion on the second signal to obtain a first full-band signal, where the first full-band signal is a full-band signal corresponding to the second signal; reconstructing the first signal based on the first full-band signal; A target sound signal is obtained based on the reconstructed first signal and the second signal.

2. The sound signal processing method according to claim 1, characterized in that: The step of performing frequency band extension on the second signal to obtain a first full-band signal includes: Based on the frequency delimiting value, filtering the original sound signal to obtain a first frequency band signal of the second signal, wherein the first frequency band signal is a signal with a frequency greater than or equal to the frequency delimiting value and less than or equal to a frequency N, where N is an adjustable frequency parameter; Performing frequency band expansion on the first frequency band signal to obtain a second full-band signal, where the second full-band signal is a full-band signal corresponding to the first frequency band signal; The second full-band signal is used as the first full-band signal.

3. The sound signal processing method according to any one of claims 1 to 2, characterized in that: The reconstructing the first signal based on the first full-band signal includes: Performing a filtering operation on the first full-band signal to obtain a second frequency band signal, wherein the second frequency band signal is a signal with a frequency greater than or equal to a frequency M and less than or equal to the frequency boundary value, and the frequency M is a signal greater than zero and less than a frequency threshold; The second frequency band signal is used as the reconstructed first signal.

4. The sound signal processing method according to claim 3, characterized in that: The step of obtaining a target sound signal based on the reconstructed first signal and the second signal includes: splicing the second signal and the reconstructed first signal to obtain a third signal; Performing spectrum envelope extraction on the original sound signal and the third signal; The target sound signal is obtained based on the spectrum envelope extraction result and the third signal.

5. The sound signal processing method according to claim 4, characterized in that: The extracting spectrum envelope of the original sound signal and the third signal comprises: The original sound signal and the third signal are transformed from the time domain to the frequency domain based on a preset signal conversion method to obtain a frequency domain signal of the original sound signal and a frequency domain signal of the third signal; Taking absolute values ​​of corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal to obtain frequency domain amplitude information of the original sound signal and frequency domain amplitude information of the third signal; Spectrum envelope extraction is performed on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

6. The sound signal processing method according to claim 5, characterized in that: The step of obtaining the target sound signal based on the spectrum envelope extraction result and the third signal includes: Obtaining an energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; Performing gain calculation on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal, and the energy adjustment value to obtain a gain calculation result; Performing spectrum envelope conversion based on the result of the gain calculation and the frequency domain signal of the third signal to obtain a frequency domain signal after spectrum envelope conversion, wherein the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value; The frequency domain signal after the spectrum envelope conversion is converted into a time domain signal based on a preset signal conversion method to obtain a target sound signal.

7. The sound signal processing method according to claim 6, characterized in that: The obtaining the energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal includes: Based on the frequency domain amplitude information of the original sound signal, obtaining the root mean square of the spectrum of the original sound signal; Obtaining a root mean square of a spectrum of the third signal based on the frequency domain amplitude information of the third signal; Obtaining an energy adjustment value of the second frequency band signal based on a ratio of a root mean square of the third signal spectrum to a root mean square of the original sound signal spectrum; The frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to a frequency boundary value.

8. A sound signal processing device, characterized in that: include: An acquisition unit, used for acquiring an original sound signal, and performing wind noise detection on the original sound signal to obtain a frequency cutoff value; a determining unit, configured to determine a first signal and a second signal based on the frequency demarcation value, wherein the first signal is used to represent a sound signal disturbed by wind noise, and the frequency value of the first signal is less than the frequency demarcation value, and the second signal is used to represent a sound signal not disturbed by wind noise, and the frequency value of the second signal is greater than or equal to the frequency demarcation value; an expansion unit, configured to perform frequency band expansion on the second signal to obtain a first full-band signal, where the first full-band signal is a full-band signal corresponding to the second signal; a reconstruction unit, configured to reconstruct the first signal based on the first full-band signal; An execution unit is used to obtain a target sound signal based on the reconstructed first signal and the second signal.

9. The sound signal processing device according to claim 8, characterized in that: The expansion unit performs frequency band expansion on the second signal to obtain a first full-band signal in the following manner: Based on the frequency delimiting value, filtering the original sound signal to obtain a first frequency band signal of the second signal, wherein the first frequency band signal is a signal with a frequency greater than or equal to the frequency delimiting value and less than or equal to a frequency N, where N is an adjustable frequency parameter; Performing frequency band expansion on the first frequency band signal to obtain a second full-band signal, where the second full-band signal is a full-band signal corresponding to the first frequency band signal; The second full-band signal is used as the first full-band signal.

10. The sound signal processing device according to any one of claims 8 to 9, characterized in that: The reconstruction unit reconstructs the first signal based on the first full-band signal in the following manner: Performing a filtering operation on the first full-band signal to obtain a second frequency band signal, wherein the second frequency band signal is a signal with a frequency greater than or equal to a frequency M and less than or equal to the frequency boundary value, and the frequency M is a signal greater than zero and less than a frequency threshold; The second frequency band signal is used as the reconstructed first signal.

11. The sound signal processing device according to claim 10, characterized in that: The execution unit obtains the target sound signal based on the reconstructed first signal and the second signal in the following manner: splicing the second signal and the reconstructed first signal to obtain a third signal; Performing spectrum envelope extraction on the original sound signal and the third signal; The target sound signal is obtained based on the spectrum envelope extraction result and the third signal.

12. The sound signal processing device according to claim 11, characterized in that: The execution unit extracts spectrum envelopes from the original sound signal and the third signal in the following manner: The original sound signal and the third signal are transformed from the time domain to the frequency domain based on a preset signal conversion method to obtain a frequency domain signal of the original sound signal and a frequency domain signal of the third signal; Taking absolute values ​​of corresponding amplitudes of the frequency domain signal of the original sound signal and the frequency domain signal of the third signal to obtain frequency domain amplitude information of the original sound signal and frequency domain amplitude information of the third signal; Spectrum envelope extraction is performed on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal to obtain the spectrum envelope of the original sound signal and the spectrum envelope of the third signal.

13. The sound signal processing device according to claim 12, characterized in that: The execution unit obtains the target sound signal based on the spectrum envelope extraction result and the third signal in the following manner: Obtaining an energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal; Performing gain calculation on the spectrum envelope of the original sound signal, the spectrum envelope of the third signal, and the energy adjustment value to obtain a gain calculation result; Performing spectrum envelope conversion on the result of the gain calculation and the frequency domain signal of the third signal to obtain a frequency domain signal after the spectrum envelope conversion, wherein the frequency range of the spectrum envelope conversion is from frequency M to a frequency boundary value; The frequency domain signal after the spectrum envelope conversion is converted into a time domain signal based on a preset signal conversion method to obtain the target sound signal.

14. The sound signal processing device according to claim 13, characterized in that: The execution unit obtains the energy adjustment value of the second frequency band signal based on the frequency domain amplitude information of the original sound signal and the frequency domain amplitude information of the third signal in the following manner: Based on the frequency domain amplitude information of the original sound signal, obtaining the root mean square of the spectrum of the original sound signal; Obtaining a root mean square of a spectrum of the third signal based on the frequency domain amplitude information of the third signal; Obtaining an energy adjustment value of the second frequency band signal based on a ratio of a root mean square of the third signal spectrum to a root mean square of the original sound signal spectrum; The frequency range of the original sound signal spectrum and the third signal spectrum is from frequency M to a frequency boundary value.

15. A sound signal processing device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the sound signal processing method described in any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to execute the sound signal processing method according to any one of claims 1 to 7.

Images