Signal processing methods and apparatus, computer-readable storage media, terminals

Abstract

A signal processing method, apparatus, computer-readable storage medium, and terminal are disclosed. The method includes: acquiring a first input signal collected by a main microphone and a second input signal collected by an auxiliary microphone; determining the user's hand gesture while holding the terminal; selecting a noise estimation algorithm based on the gesture, and using the selected noise estimation algorithm to estimate the noise in the first input signal and / or the second input signal to obtain a noise signal; and processing the first input signal or the second input signal based on the noise signal to obtain an output signal. This application provides a more robust signal processing method, which is beneficial for improving the quality of the speech signal output by the terminal.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, and in particular to a signal processing method and apparatus, a computer-readable storage medium, and a terminal. Background Technology

[0002] Currently, users have increasingly higher requirements for voice communication quality. To address this, some existing terminals are equipped with at least two microphones to improve noise suppression. For example, in a scenario with two microphones, one microphone is used to capture background noise, and the other is used to capture noisy speech. Noise reduction is then performed based on the signals from both microphones. However, existing noise reduction schemes have poor robustness, and poor voice quality can still easily occur during hands-on calls. Summary of the Invention

[0003] One of the technical objectives of this application is to provide a more robust signal processing method to improve the voice quality of the terminal.

[0004] To address the aforementioned technical problems, this application provides a signal processing method applied to a terminal equipped with a main microphone and an auxiliary microphone. The method includes: acquiring a first input signal collected by the main microphone and a second input signal collected by the auxiliary microphone; determining the user's hand gesture on the terminal; selecting a noise estimation algorithm based on the gesture, and using the selected noise estimation algorithm to perform noise estimation on the first input signal and / or the second input signal to obtain a noise signal; and processing the first input signal or the second input signal based on the noise signal to obtain an output signal.

[0005] Optionally, the user's hand position on the terminal is related to the angle between the terminal and the vertical direction; wherein, the larger the angle, the greater the distance between the main microphone and the user's mouth.

[0006] Optionally, the posture belongs to a first posture. In the first posture, the included angle is greater than or equal to a first preset value. The noise estimation algorithm selected is used to perform noise estimation on the first input signal and / or the second input signal to obtain the noise signal. This includes: using a single-microphone noise estimation algorithm to perform noise estimation on the first input signal or the second input signal to obtain the noise signal.

[0007] Optionally, the posture belongs to a first posture. In the first posture, the included angle is greater than or equal to a first preset value. The noise estimation algorithm selected is used to perform noise estimation on the first input signal and / or the second input signal to obtain the noise signal, which includes: filtering the first input signal according to the second input signal to obtain the noise signal.

[0008] Optionally, performing noise estimation on the first input signal and / or the second input signal using a selected noise estimation algorithm to obtain a noise signal includes: filtering the second input signal based on the first input signal to obtain the noise signal; wherein the update rate of the filter coefficients used in the filtering process is positively correlated with the included angle.

[0009] Optionally, the update rate of the filter coefficients in the second posture is greater than the update rate of the filter coefficients in the third posture; wherein, in the second posture, the included angle is greater than a second preset value, and in the third posture, the included angle is less than or equal to the second preset value.

[0010] Optionally, the posture belongs to the second posture. Before filtering the second input signal according to the first input signal, the method further includes: determining whether there is speech activity in the current frame; if there is speech activity in the current frame, updating the filtering coefficients according to the first step length value; otherwise, updating the filtering coefficients according to the second step length value, wherein the second step length value is less than the first step length value.

[0011] Optionally, the second step size is 0.

[0012] This application embodiment also provides a signal processing device, the device comprising: an acquisition module, configured to acquire a first input signal collected by the main microphone of a terminal and a second input signal collected by an auxiliary microphone; a posture determination module, configured to determine the posture of a user holding the terminal; a noise estimation module, configured to select a noise estimation algorithm according to the posture, and use the selected noise estimation algorithm to perform noise estimation on the first input signal and / or the second input signal to obtain a noise signal; and a processing module, configured to process the first input signal or the second input signal according to the noise signal to obtain an output signal.

[0013] This application also provides a computer-readable storage medium storing a computer program thereon, which, when run by a processor, performs the steps of the above-described signal processing method.

[0014] This application embodiment also provides a terminal, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the steps of the above-described signal processing method when running the computer program.

[0015] Compared with the prior art, the technical solution of this application embodiment has the following beneficial effects:

[0016] In the scheme of this embodiment, the user's handheld terminal posture is determined, and a noise estimation algorithm is selected based on the user's handheld terminal posture. Then, the selected noise estimation algorithm is used to estimate the noise of the first input signal collected by the main microphone and / or the second input signal collected by the auxiliary microphone to obtain a noise signal. The first input signal or the second input signal is processed based on the noise signal to obtain an output signal. In this embodiment, the noise estimation algorithm is selected based on the user's handheld terminal posture, so that the algorithm used for noise estimation is adapted to the user's handheld terminal posture. This allows for better noise suppression under different postures, achieving stable noise reduction performance and better robustness in noise reduction.

[0017] Furthermore, in the solution of this application embodiment, when the included angle corresponding to the user's handheld terminal posture is greater than or equal to a first preset value, a single-microphone noise estimation algorithm is used for noise estimation, or the first input signal is filtered using a second input signal to obtain a noise signal. This solution can obtain a more accurate noise signal even when the terminal has a large external range, which helps to avoid speech distortion after noise reduction.

[0018] Furthermore, in the scheme of this application embodiment, the second input signal is filtered based on the first input signal to obtain a noise signal, wherein the update speed of the filter coefficients under the second posture is greater than the update speed of the filter coefficients under the third posture. This scheme can accelerate the update speed of the filter coefficients when the terminal is extended, eliminate more speech in the second input signal, obtain a more accurate noise signal, and help reduce speech loss during the noise reduction process.

[0019] Furthermore, in the scheme of this application embodiment, under the second posture, before filtering the second input signal based on the first input signal, it is determined whether there is speech activity in the current frame; if there is speech activity in the current frame, the filtering coefficients are updated according to the first step length value; otherwise, the filtering coefficients are updated according to the second step length value, wherein the second step length value is less than the first step length value. This scheme helps to improve the situation where noise is too low due to faster updating of the filtering coefficients, making the estimated noise signal more accurate and improving the noise reduction performance. Attached Figure Description

[0020] Figure 1 This is a diagram illustrating the posture of a user holding a terminal.

[0021] Figure 2 This is a schematic diagram of the structure of a terminal according to an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the structure of another terminal in the embodiments of this application;

[0023] Figure 4 This is a schematic flowchart of a signal processing method according to an embodiment of this application;

[0024] Figure 5 This is a diagram illustrating another user's handheld terminal posture;

[0025] Figure 6 This is a flowchart illustrating another signal processing method in an embodiment of this application;

[0026] Figure 7 This is a performance comparison diagram of signal processing using the scheme of the present application embodiment and using the existing scheme;

[0027] Figure 8 This is a comparative diagram of the performance of signal processing using the scheme of the present application embodiment and the existing scheme;

[0028] Figure 9 This is a schematic diagram of the structure of a signal processing device according to an embodiment of this application. Detailed Implementation

[0029] As described in the background section, existing noise reduction solutions are not very robust, and poor voice quality is still likely to occur during user handheld terminal calls.

[0030] Existing noise reduction schemes typically only consider normal posture (e.g., Figure 1 The scene depicting the pose. (Refer to...) Figure 1 , Figure 1 This is a diagram illustrating the posture of a user holding a terminal.

[0031] like Figure 1 As shown, Figure 1 This illustrates the typical handholding posture of a user. Specifically, in Figure 1 In the shown posture, the main microphone is close to the user's mouth, and the distance between the main microphone and the user's mouth is significantly smaller than the distance between the auxiliary microphone and the user's mouth. There is a significant energy difference between the signals collected by the main microphone and the signals collected by the auxiliary microphone. Using the energy difference between the two for noise estimation can effectively suppress noise.

[0032] However, due to user habits and other factors, in practical applications, users are likely to use the speaker with their hands. When using the speaker with their hands, the distance between the main microphone and the user's mouth is relatively large, and the signal energy picked up by the main microphone and the auxiliary microphone is similar. If the noise estimation method used under normal posture is applied, speech loss or even complete silence may occur.

[0033] To address the aforementioned technical problems, this application provides a signal processing method. In this embodiment, the user's handheld terminal posture is determined, and a noise estimation algorithm is selected based on the posture. Then, the selected noise estimation algorithm is used to estimate the noise of a first input signal acquired by the main microphone and / or a second input signal acquired by the auxiliary microphone, resulting in a noise signal. The first or second input signal is then processed based on the noise signal to obtain an output signal. In this embodiment, the noise estimation algorithm is selected based on the user's handheld terminal posture, making the algorithm suitable for the user's handheld terminal posture. This allows for better noise suppression under different postures, achieving higher noise reduction performance and better robustness.

[0034] To make the above-mentioned objectives, features and beneficial effects of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0035] It should be noted that the terminal in this embodiment may be configured with at least two microphones, and the microphones may refer to a voice acquisition module, which is integrated into the terminal. The terminal may be a voice call device, a voice recording device, etc., such as a mobile phone, wearable device, computer, tablet computer, etc., but is not limited thereto. The voice acquisition module may also be any existing device with voice acquisition function; this embodiment does not limit the structure or model of the voice acquisition module.

[0036] Furthermore, the terminal may be equipped with at least two microphones, including a main microphone and an auxiliary microphone. The main microphone and the auxiliary microphone are located in different positions on the terminal. Ideally, the main microphone picks up the user's voice, while the auxiliary microphone picks up background noise.

[0037] Reference Figure 2 , Figure 2 This is a schematic diagram of the structure of a terminal in an embodiment of this application.

[0038] like Figure 2 As shown, the main microphone 101 is located at the bottom of the terminal 10, and the auxiliary microphone 102 is located at the top of the terminal 10.

[0039] Reference Figure 3 , Figure 3 This is a schematic diagram of the structure of another terminal in the embodiments of this application.

[0040] like Figure 3As shown, the main microphone 201 is located at the bottom of the terminal 20, and the auxiliary microphone 202 is located at the back of the terminal 20. More specifically, the auxiliary microphone 202 is located in the upper area of ​​the back of the terminal 20. In a specific example, the auxiliary microphone 202 may be located near the rear camera of the terminal 20.

[0041] It should be noted that, Figure 2 and Figure 3 The layout of the main microphone and auxiliary microphone on the terminal is shown only as an example. In actual applications, the main microphone and auxiliary microphone can also be set in other locations on the terminal, and this embodiment is not limited to this.

[0042] Reference Figure 4 , Figure 4 This is a flowchart illustrating a signal processing method according to an embodiment of this application. The method can be executed by a terminal equipped with a main microphone and an auxiliary microphone; in other words, the terminal is a dual-channel device. For a detailed description of the terminal, please refer to the relevant description above, which will not be repeated here.

[0043] Figure 4 The signal processing method shown may include the following steps:

[0044] Step S41: Acquire the first input signal collected by the main microphone and the second input signal collected by the auxiliary microphone;

[0045] Step S42: Determine the user's hand position when holding the terminal;

[0046] Step S43: Select a noise estimation algorithm according to the posture, and use the selected noise estimation algorithm to estimate the noise of the first input signal and / or the second input signal to obtain a noise signal;

[0047] Step S44: Process the first input signal or the second input signal according to the noise signal to obtain the output signal.

[0048] It is understandable that, in specific implementation, the above method can be implemented using a software program that runs in the processor integrated inside the chip or chip module; or, the method can be implemented using hardware or a combination of hardware and software, such as using a dedicated chip or chip module, or using a dedicated chip or chip module combined with a software program.

[0049] In the specific implementation of step S41, a first input signal and a second input signal are acquired. The first input signal is acquired by the main microphone of the terminal, and the second input signal is acquired by the auxiliary microphone of the terminal. For ease of description, the main microphone can also be referred to as the "main microphone" and the auxiliary microphone as the "auxiliary microphone". The first input signal can be represented as s1(n) and the second input signal can be represented as s2(n). Both the first input signal s1(n) and the second input signal s2(n) are time-domain signals.

[0050] In the specific implementation of step S42, the user's handheld terminal posture can be determined. In this embodiment, the user's handheld terminal posture is related to the angle between the terminal and the vertical direction. The vertical direction can refer to the direction perpendicular to the horizontal plane.

[0051] Specifically, the angle between the terminal and the vertical direction varies depending on how the user holds the terminal. The size of this angle can be used to characterize the degree to which the terminal extends outward.

[0052] Reference Figure 5 , Figure 5 This is a schematic diagram of another user handheld terminal posture in an embodiment of this application. Figure 5 This illustrates a scenario where the terminal is expanded externally. Figure 5 In the illustrated position, the main microphone is far from the user's mouth, and the difference between the distance between the main microphone and the user's mouth, and the distance between the auxiliary microphone and the user's mouth, is relatively small. In some cases, the distance between the auxiliary microphone and the user's mouth may be greater than the distance between the main microphone and the user's mouth.

[0053] like Figure 5 As shown, the angle between the terminal and the vertical direction is θ. The size of angle θ is positively correlated with the degree of terminal expansion. The larger θ is, the greater the distance between the main microphone and the user's mouth, resulting in less voice signal and more noise signal in the first input signal. Furthermore, compared to... Figure 1 In the scenario shown in the normal posture diagram, the proportion of speech signal in the second input signal will increase in the case of an externally extended terminal. More specifically, the larger θ is, the more speech signal is in the second input signal.

[0054] In practice, multiple postures can be preset, each with a corresponding angular range. These angular ranges for each preset posture are distinct or non-overlapping. More specifically, the user's handheld terminal posture can be determined from the preset postures using any of the following methods, but is not limited to these:

[0055] Method 1: Direction of Arrival (DOA) can be estimated based on the phase difference between the first and second input signals to determine the direction of the sound source and ultimately the user's handheld terminal posture.

[0056] Method 2: The user's handheld terminal posture can be determined based on the energy difference between the first and second input signals. Specifically, the energy difference between the first and second input signals varies under different postures. The greater the user's outward extension, the smaller the energy difference between the first and second input signals. Therefore, the user's handheld terminal posture can be determined based on the energy difference between the first and second input signals.

[0057] Method 3: Sensor-based methods can be used to determine the user's handheld terminal posture. For example, the terminal can be equipped with a proximity sensor to detect the user's handheld terminal posture.

[0058] It should be noted that the above is only an exemplary implementation of determining the posture of the user's handheld terminal. In practical applications, other appropriate methods can also be used to determine the posture of the user's handheld terminal. Different postures result in different degrees of terminal expansion.

[0059] In a non-limiting example, the aforementioned angle θ can be determined and used as the user's handheld terminal posture. That is, the user's handheld terminal posture can be represented by the angle θ.

[0060] In practice, step S42 can be executed periodically at preset time intervals. Specifically, during the process of the main microphone acquiring the first input signal and the auxiliary microphone acquiring the second input signal, the user's handheld terminal posture can be determined periodically. This approach allows for the identification of changes in the user's posture during a call, thereby enabling dynamic switching of the noise estimation algorithm and helping to maintain good call quality throughout the call.

[0061] Continue to refer to Figure 4 In the specific implementation of step S43, a noise estimation algorithm can be selected based on the posture of the user's handheld terminal determined in step S42, and the selected noise estimation algorithm can be used to perform noise estimation.

[0062] In one embodiment of this application, if the user's handheld terminal posture is a first posture, noise estimation can be performed using method one or method two, but it is not limited to these. In the first posture, the included angle θ is greater than or equal to a first preset value. The first posture in this embodiment can be understood as a posture with a large degree of outward expansion. In the first posture, the proportion of voice signal in the first input signal collected by the main microphone is small, and the proportion of noise signal is large.

[0063] Method 1: Use a single-microphone noise estimation algorithm to estimate the noise of the first or second input signal to obtain the noise signal.

[0064] In practice, a single-microphone noise estimation algorithm can be used to estimate the noise of the first input signal of the current frame to obtain the noise signal of the current frame. Alternatively, a single-microphone noise estimation algorithm can be used to estimate the noise of the second input signal of the current frame to obtain the noise signal of the current frame.

[0065] The single-microphone noise estimation algorithm can be any existing appropriate algorithm, such as the minimum tracking method, the recursive averaging method, etc. This embodiment does not limit it.

[0066] Method 2: Filter the first input signal based on the second input signal to obtain a noise signal.

[0067] Specifically, the second input signal can be delayed to obtain a delayed signal; further, the estimated speech signal can be determined based on the delayed signal and the filtering coefficients, that is, the speech signal can be estimated based on the signal collected by the auxiliary microphone; then the first input signal can be filtered based on the estimated speech signal to obtain a noise signal.

[0068] More specifically, if the user's handholding posture is the first posture, the noise signal can be determined according to the following formula:

[0069]

[0070] Where s4(n) represents the noise signal, s3(n) represents the delayed signal, and h(n) represents the time-domain filtering coefficients.

[0071] It should be noted that n represents the sampling point index within each frame of speech, and the temporal filtering coefficients can be updated frame by frame. That is, the temporal filtering coefficients can be updated one frame at a time.

[0072] Furthermore, if the user's handheld terminal posture does not belong to the first posture, the second input signal can be filtered based on the first input signal to obtain a noise signal. If the user's handheld terminal posture does not belong to the first posture, it indicates that the amplification is relatively small, and correspondingly, the proportion of voice signal in the first input signal collected by the main microphone is relatively large.

[0073] Specifically, the first input signal can be delayed to obtain a delayed signal; further, the estimated speech signal can be determined based on the delayed signal and the filtering coefficients, that is, the speech signal can be estimated based on the signal collected by the main microphone; then the second input signal can be filtered based on the estimated speech signal to obtain a noise signal.

[0074] More specifically, if the user's hand gesture is not the first gesture, the noise signal can be determined according to the following formula:

[0075]

[0076] In another embodiment of this application, the second input signal can be filtered based on the first input signal to obtain a noise signal. The filtering coefficients h(n) used in the filtering process can be continuously updated in the time domain, and the update rate of the filtering coefficients is positively correlated with the included angle. That is, the larger the included angle, the faster the update rate of the filtering coefficients. In other words, in this embodiment, the update rate of the filtering coefficients is used to adapt to different postures; that is, the update rate of the filtering coefficients is adjusted to adapt to different degrees of outward expansion.

[0077] In a specific example, the update rate of the filter coefficients in the second posture is greater than that in the third posture. Specifically, in the second posture, the included angle θ is greater than a second preset value, while in the third posture, the included angle θ is less than or equal to the second preset value. That is, the expansion degree in the second posture is greater than that in the third posture.

[0078] In practice, the filter coefficients of the current frame can be obtained by updating the filter coefficients of the previous frame based on the step size value. Therefore, the update speed of the filter coefficients can be adjusted by adjusting the step size value used to update the filter coefficients.

[0079] Specifically, if the pose corresponding to the current frame is the second pose, the filter coefficients of the previous frame can be updated using the first step length value. If the pose corresponding to the current frame is the third pose, the filter coefficients of the previous frame can be updated using the second step length value, where the second step length value is less than the first step length value.

[0080] In a non-restrictive example, if the user's hand gesture is a second gesture, then before filtering the second input signal of the current frame based on the first input signal of the current frame, it can be determined whether there is voice activity in the current frame. If so, the filtering coefficients of the previous frame can be updated based on the first step length value; otherwise, the filtering coefficients of the previous frame can be updated based on the second step length value, where the second step length value is less than the first step length value.

[0081] Specifically, the presence of speech activity in the current frame can be determined using the Voice Active Detection (VAD) method. More specifically, a single-microphone VAD method can be used. This embodiment does not limit the single-microphone VAD method; for example, it can be a pitch detection-based VAD method, a deep learning-based VAD method, etc.

[0082] In a specific example, the first input signal of the current frame can be subjected to single-microphone noise reduction. If the energy difference between the input and output signals before and after single-microphone noise reduction is greater than a preset threshold, it can be determined that there is no speech activity in the current frame; otherwise, it can be determined that there is speech activity in the current frame.

[0083] Furthermore, if there is speech activity in the current frame, the filter coefficients are updated using the first step long value to speed up the update process; if there is no speech activity in the current frame, the filter coefficients do not need to be updated, meaning the filter coefficients from the previous frame can be used as the filter coefficients for the current frame. In other words, the second step long value is 0.

[0084] In practical implementation, considering the possibility of errors in voice activity detection, failure to update the filter coefficients may result in inaccurate noise estimation. Therefore, in this embodiment, the second step size can be greater than 0 and less than the first step size. For example, the second step size can be half of the first step size.

[0085] As described above, the noise signal can be obtained by executing steps S42 and S43. Since the noise signal is determined by a noise estimation algorithm adapted to the user's handheld terminal posture, the obtained noise signal is more accurate.

[0086] It should be noted that in this embodiment, an adaptive filter can be used for filtering. Specifically, the input signal of the adaptive filter is a delayed first input signal, and the reference signal is a second input signal; or the input signal of the adaptive filter is a delayed second input signal, and the reference signal is the first input signal. The output signal of the adaptive filter is a noise signal. This embodiment does not limit the update algorithm used for the adaptive filter. For example, it can be any of the following, but is not limited to: Least Mean Square (LMS), Normalized Least Mean Square (NLMS), Recursive Least Square (RLS), Frequency Domain Least Mean Square, Windowed Frequency Domain Least Mean Square, etc.

[0087] It should also be noted that in this embodiment, the delay can be applied based on the distance between the main microphone and the auxiliary microphone. More specifically, the delay level is delay = d / c, where d represents the distance between the main microphone and the auxiliary microphone, and c represents the speed of sound.

[0088] In the specific implementation of step S44, post-processing can be performed on the noise signal to obtain the output signal, which can refer to the signal obtained after noise reduction.

[0089] Specifically, if the noise signal is obtained by filtering the second input signal or by using a single-microphone noise estimation algorithm, then in step S44, the first input signal can be post-processed based on the noise signal to obtain the output signal. If the noise signal is obtained by filtering the first input signal, then in step S44, the second input signal can be post-processed based on the noise signal to obtain the output signal.

[0090] In a non-limiting example, when the pose belongs to the first pose, the first input signal is filtered based on the second input signal to obtain a noise signal; then the second input signal is post-processed to obtain the output signal. When the pose does not belong to the first pose (e.g., the second or third pose), the second input signal is filtered based on the first input signal to obtain a noise signal; then the first input signal is post-processed to obtain the output signal.

[0091] In practical applications, the extent of terminal expansion during a user's call may gradually increase. When a switch from a posture other than the first posture (e.g., the second posture) to the first posture is detected, if the above-mentioned scheme is adopted, a sudden change in volume may occur, that is, a sudden change in the input signal. To improve this problem, embodiments of this application further propose that: when using method two for noise estimation, before processing the second input signal based on the noise signal, adaptive gain adjustment can be performed first, and then the second input signal after adaptive gain adjustment can be processed based on the noise signal to obtain the output signal.

[0092] Specifically, the adaptive gain can be calculated using the following formula:

[0093]

[0094] Where λ is the frame index of the speech frame, i is the frequency index, Sa1(λ,i) is the amplitude spectrum of the first frequency domain signal corresponding to the first input signal at the i-th frequency point, Sa2(λ,i) is the amplitude spectrum of the second frequency domain signal corresponding to the second input signal at the i-th frequency point; P(λ,i) is the speech presence probability at the i-th frequency point of the λ-th frame, α is the gain smoothing coefficient between 0 and 1, which can generally be selected from 0.95 to 0.98, noisefloor is the amplitude spectrum of the noise floor, which can generally be selected from -80dB, FS is the starting frequency point for calculating the sum of amplitude spectra, and FE is the ending frequency point for calculating the sum of amplitude spectra, which is generally selected from 200Hz to 4000Hz.

[0095] The greater the extent of terminal expansion, the closer the value of gain(λ) of the voice frame is to 0, while the value of gain(λ) of the noise frame is close to 1 because the value of P(λ,i) is very small.

[0096] Furthermore, the product of the adaptive gain of the λth frame and the second input signal of the λth frame can be calculated, and the product can be processed according to the noise signal to obtain the final output signal.

[0097] As shown above, processing the second input signal according to the adaptive gain helps to solve the problem of sudden changes in the output signal volume.

[0098] Therefore, in the solution of this application embodiment, the noise estimation algorithm is selected according to the posture of the user's handheld terminal, so that the algorithm used for noise estimation is adapted to the posture of the user's handheld terminal, thereby suppressing noise well under different postures, obtaining higher noise reduction performance, and the solution has better robustness.

[0099] Reference Figure 6 , Figure 6 This is a flowchart illustrating another signal processing method in an embodiment of this application. The following mainly focuses on... Figure 6 and Figure 4 The differences will be explained.

[0100] Step S61: Obtain the first input signal and the second input signal.

[0101] Step S62: Determine the posture of the user's handheld terminal in the current frame.

[0102] In practical implementation, a first posture, a second posture, and a third posture are preset. In the first posture, the included angle is greater than or equal to a first preset value; in the second posture, the included angle is less than the first preset value but greater than the second preset value; and in the third posture, the included angle is less than or equal to the second preset value, wherein the first preset value is greater than the second preset value. The specific values ​​of the first and second preset values ​​can be set according to actual application requirements.

[0103] In a specific example, the second preset value is 30 degrees, and the first preset value is 90 degrees. If the user holds the terminal in the first posture, the degree of terminal expansion can be determined to be large; this first posture can also be called "large-angle expansion." If the user holds the terminal in the second posture, the degree of terminal expansion can be determined to be relatively large; this second posture can also be called "moderate expansion." If the user holds the terminal in the third posture, the degree of terminal expansion can be determined to be relatively small, close to... Figure 1 The posture shown, the third posture, can also be called the "normal posture".

[0104] If it is determined that the user's handheld terminal's posture in the current frame belongs to the first posture, then proceed to step S63; if it is determined that the user's handheld terminal's posture in the current frame belongs to the second posture, then proceed to step S64; if it is determined that the user's handheld terminal's posture in the current frame belongs to the third posture, then proceed to step S65.

[0105] Step S63: Use a single-microphone noise estimation algorithm to estimate the noise of the first input signal or the second input signal to obtain the noise signal of the current frame.

[0106] Step S64: Determine whether there is voice activity in the current frame. If the determination result is yes, proceed to step S66; otherwise, proceed to step S65.

[0107] Step S65: Update the filter coefficients of the previous frame based on the long value from the first step to obtain the filter coefficients of the current frame.

[0108] Step S66: Update the filter coefficients of the previous frame according to the second step length value to obtain the filter coefficients of the current frame.

[0109] In a specific example, the adaptive filter update algorithm uses NLMS, and the update strategy for the filter coefficients is as follows:

[0110]

[0111] Where H is the frequency domain representation of the filter coefficients of the adaptive filter, H(λ) is the frequency domain representation of the filter coefficients of the current frame, which is a 1×N vector, μ(λ) represents the step size of the current frame, ΔH represents the update term, λ represents the frame index, FFT represents the Fast Fourier Transform, IFFT represents the Inverse Fast Fourier Transform, M represents the number of sampling points per frame, N represents the number of FFT points per frame, and X is the frequency domain representation of the second input signal. * Let X be the conjugate of X, E be the frequency domain representation of the error signal, δ be the preset parameter used to prevent the filter from diverging, and P(λ) be the speech probability vector of the current frame, which is the speech probability vector of the first to Nth frequency points of the current frame. When there is speech, the speech probability of each frequency point (i.e., the first to Nth frequency points) approaches 1. When there is no speech, the speech probability of each frequency point approaches 0, Δh also approaches 0, and the filter coefficients are hardly updated.

[0112] If the pose is the third pose, or if the pose is the second pose and there is no speech activity in the current frame, the filter coefficients of the current frame can be updated according to H(λ)=H(λ-1)+μ(λ-1)×ΔH(λ); if the pose is the second pose and there is speech activity in the current frame, the filter coefficients of the current frame can be updated according to H(λ)=H(λ-1)+2×μ(λ-1)×ΔH(λ).

[0113] Step S67: Based on the filtering coefficients of the current frame and the first input signal of the current frame, filter the second input signal of the current frame to obtain the noise signal of the current frame.

[0114] Step S68: Process the first input signal of the current frame according to the noise signal of the current frame to obtain the output signal of the current frame.

[0115] about Figure 6 For more information, please refer to Figure 4 The relevant descriptions in the document will not be repeated here.

[0116] Reference Figure 7 and Figure 8 , Figure 7 This is a schematic diagram comparing the performance of signal processing using the scheme of this application embodiment and the scheme using existing methods. Figure 8 This is a schematic diagram comparing the performance of the scheme using the embodiments of this application and the signal processing using existing schemes.

[0117] More specifically, Figure 7 and Figure 8 It shows the use of Figure 6 The proposed solution is shown in comparison with the noise reduction performance of other existing solutions. Figure 7 This is a performance comparison under the first posture described above. Figure 8 This is a performance comparison under the second posture described above.

[0118] like Figure 7 As shown, in the first posture, other existing solutions will result in a silent phenomenon. The solution provided in this embodiment can improve the silent problem under large-angle expansion. For details, please refer to... Figure 7 The portion indicated by the middle frame 72.

[0119] like Figure 8 As shown, in the second posture, other existing solutions will result in speech distortion, especially severe speech distortion in the high-frequency range. The solution provided in this embodiment can significantly improve the speech distortion problem under general external amplification. For details, please refer to... Figure 8 The portions indicated by boxes 81 and 82.

[0120] Reference Figure 9 , Figure 9 This is a schematic diagram of the structure of a signal processing device according to an embodiment of this application. Figure 9 The apparatus shown may include:

[0121] The acquisition module 91 is used to acquire the first input signal collected by the main microphone of the terminal and the second input signal collected by the auxiliary microphone;

[0122] The posture determination module 92 is used to determine the posture in which the user holds the terminal;

[0123] The noise estimation module 93 is used to select a noise estimation algorithm according to the posture, and use the selected noise estimation algorithm to perform noise estimation on the first input signal and / or the second input signal to obtain a noise signal;

[0124] The processing module 94 is used to process the first input signal or the second input signal according to the noise signal to obtain an output signal.

[0125] In practice, Figure 9 The signal processing device shown may correspond to a chip with signal processing function in a terminal; or to a chip module with signal processing function; or to a terminal.

[0126] For more information on the working principle, working method, and beneficial effects of the signal processing device in the embodiments of this application, please refer to the relevant description of the signal processing method above, which will not be repeated here.

[0127] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program is run by a processor, it performs the steps of the signal processing method described above. The storage medium may include ROM, RAM, a magnetic disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0128] This application also provides a terminal, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it performs the steps of the signal processing method described above. The terminal includes, but is not limited to, terminal devices such as mobile phones, computers, and tablet computers.

[0129] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0130] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0131] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatus, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or in a combination of hardware and software functional units. For example, for various devices or products applied to or integrated into a chip, each module / unit can be implemented using hardware such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware such as circuits; for various devices or products applied to or integrated into a chip module, each module / unit can be implemented using hardware such as circuits, and different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware such as circuits. The components can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, each of its components / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or in different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.

[0134] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0135] In this application's embodiments, "multiple" refers to two or more. The descriptions of "first," "second," etc., appearing in this application's embodiments are merely illustrative and for distinguishing the described objects; they do not indicate any order and do not imply a specific limitation on the number of devices in this application's embodiments, nor do they constitute any limitation on the embodiments of this application. Although this application discloses the above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

[0136] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. A signal processing method, characterized by, The method is applied to a terminal configured with a main microphone and an auxiliary microphone, and comprises: obtaining a first input signal collected by the main microphone and a second input signal collected by the auxiliary microphone; determining a posture in which a user holds the terminal; selecting a noise estimation algorithm according to the posture, and performing noise estimation on the first input signal and / or the second input signal by using the selected noise estimation algorithm to obtain a noise signal; processing the first input signal or the second input signal according to the noise signal to obtain an output signal; wherein the posture in which the user holds the terminal is related to an included angle between the terminal and a vertical direction, and the included angle can be used to represent a degree of expansion of the terminal; wherein the larger the included angle, the greater the distance between the main microphone and the user's mouth; wherein, before processing the second input signal according to the noise signal, adaptive gain adjustment is performed first, and then the second input signal after the adaptive gain adjustment is processed according to the noise signal to obtain the output signal, and the adaptive gain is calculated according to the following formula: ； in, The frame index for the speech frame. For frequency point index, The first frequency domain signal corresponding to the first input signal is in the first frequency domain. The amplitude spectrum at each frequency point The second frequency domain signal corresponding to the second input signal is in the second frequency domain. The amplitude spectrum of each frequency point; For the first Frame number The probability of speech at each frequency point exists. is the gain smoothing coefficient between 0 and 1, noisefloor is the amplitude spectrum of the noise floor, FS is the starting frequency for calculating the sum of amplitude spectra, and FE is the ending frequency for calculating the sum of amplitude spectra.

2. The signal processing method of claim 1, wherein, the posture belongs to a first posture, and in the first posture, the included angle is greater than or equal to a first preset value, and the noise estimation on the first input signal and / or the second input signal by using the selected noise estimation algorithm to obtain the noise signal comprises: performing noise estimation on the first input signal or the second input signal by using a single-microphone noise estimation algorithm to obtain the noise signal.

3. The signal processing method of claim 1, wherein, the posture belongs to a first posture, and in the first posture, the included angle is greater than or equal to a first preset value, and the noise estimation on the first input signal and / or the second input signal by using the selected noise estimation algorithm to obtain the noise signal comprises: filtering processing is performed on the first input signal according to the second input signal to obtain the noise signal.

4. The signal processing method of claim 1, wherein, the noise estimation on the first input signal and / or the second input signal by using the selected noise estimation algorithm to obtain the noise signal comprises: filtering processing is performed on the second input signal according to the first input signal to obtain the noise signal; wherein the update speed of a filter coefficient used in the filtering processing is positively related to the included angle.

5. The signal processing method of claim 4, wherein, the update speed of the filter coefficient in a second posture is greater than the update speed of the filter coefficient in a third posture; wherein, in the second posture, the included angle is greater than a second preset value, and in the third posture, the included angle is less than or equal to the second preset value.

6. The signal processing method of claim 5, wherein, the posture belongs to the second posture, and before the filtering processing is performed on the second input signal according to the first input signal, the method further comprises: determining whether there is voice activity in a current frame; if there is voice activity in the current frame, the filter coefficient is updated according to a first step value, otherwise, the filter coefficient is updated according to a second step value, wherein the second step value is less than the first step value.

7. The signal processing method of claim 6, wherein, the second step value is 0.

8. A signal processing device, characterized by The apparatus comprises: an obtaining module configured to obtain a first input signal collected by a main microphone of a terminal and a second input signal collected by an auxiliary microphone of the terminal; a posture determining module configured to determine a posture in which a user holds the terminal; a noise estimation module, configured to select a noise estimation algorithm according to the posture, and to perform noise estimation on the first input signal and / or the second input signal by using the selected noise estimation algorithm to obtain a noise signal; a processing module, configured to process the first input signal or the second input signal according to the noise signal to obtain an output signal; wherein the posture of the user holding the terminal is related to an included angle between the terminal and a vertical direction, and the included angle can be used to represent a degree of expansion of the terminal; wherein the greater the included angle, the greater the distance between the main microphone and the mouth of the user; wherein the processing module, before processing the second input signal according to the noise signal, first performs adaptive gain adjustment, and then processes the second input signal after the adaptive gain adjustment according to the noise signal to obtain the output signal, and the adaptive gain is calculated according to the following formula: ； wherein, is a frame index of the speech frame, is a frequency bin index, is an amplitude spectrum of a first frequency domain signal corresponding to the first input signal at the th frequency bin, is an amplitude spectrum of a second frequency domain signal corresponding to the second input signal at the th frequency bin; is a speech presence probability of the th frequency bin of the th frame, is a gain smoothing factor between 0 and 1, noisefloor is an amplitude spectrum of a noise floor, FS is a starting frequency bin for computing the amplitude spectrum sum, and FE is an ending frequency bin for computing the amplitude spectrum sum.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is run by the processor, and the steps of the signal processing method in any one of claims 1 to 7 are executed.

10. A terminal comprising a memory and a processor, said memory having stored thereon a computer program that is operable on said processor, characterized in that, The processor runs the computer program, and the steps of the signal processing method in any one of claims 1 to 7 are executed.

Images