Voice pickup method and device, electronic equipment and computer readable storage medium
By calculating the instantaneous phase difference and probability of the audio signal in the microphone array, the speech spectrum gain is generated, and the problem of poor sound pickup performance in the prior art is solved, effectively suppressing environmental interference and improving speech clarity is achieved.
Patent Information
- Application Number
- CN202411154780.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, the microphone array has poor sound pickup performance in a noisy environment, and cannot effectively suppress interference noise, affecting the clarity of the target voice.
By obtaining the audio signals collected by the two microphones, converting them into frequency domain signals, instantaneous phase difference is calculated, posterior probability is calculated based on the prior probability and the synthetic signal, the speech spectrum gain is generated, and the target audio signal is generated to suppress environmental interference.
Improve sound pickup performance, accurately determine the probability of voice existence, effectively suppress environmental interference, and improve the clarity of voice pickup.
Smart Images

Figure CN118918917A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of sound pickup, and in particular to a voice pickup method, device, electronic equipment and computer-readable storage medium. Background Art
[0002] Extracting clean target speech from a noisy environment is a widely studied topic. In the prior art, microphone arrays are usually used to obtain significant interference noise suppression effects while ensuring that the target speech is not distorted; however, due to structural layout, lightweight, cost, routing and other factors, the actual setting of the microphone array cannot meet the theoretical requirements, affecting the sound pickup performance. Summary of the invention
[0003] The main purpose of the present invention is to provide a voice pickup method, device, electronic device and computer-readable storage medium, aiming to solve the problem of poor sound pickup performance in the prior art.
[0004] To achieve the above object, the present invention provides a voice pickup method, the method comprising the steps of:
[0005] Acquire a first audio signal and a second audio signal, and convert the first audio signal into a first frequency domain signal, and convert the second audio signal into a second frequency domain signal; wherein the first audio signal is collected by a first microphone, and the second audio signal is collected by a second microphone;
[0006] Calculating an instantaneous phase difference between the first frequency domain signal and the second frequency domain signal;
[0007] Calculating a priori probability of speech presence in the audio signal based on the instantaneous phase difference;
[0008] synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal;
[0009] Calculate the posterior probability of the presence of speech in the audio signal based on the prior probability and the synthesized signal;
[0010] Calculate the speech spectrum gain according to the posterior probability;
[0011] A target audio signal is generated according to the speech spectrum gain and the synthesized signal.
[0012] Optionally, calculating the instantaneous phase difference between the first microphone and the second microphone according to the first frequency domain signal and the second frequency domain signal includes:
[0013] Calculate a first instantaneous phase angle of the first microphone according to the first frequency domain signal, and calculate a second instantaneous phase angle of the second microphone according to the second frequency domain signal;
[0014] The difference between the first instantaneous phase angle and the second instantaneous phase angle is calculated to obtain the instantaneous phase difference.
[0015] Optionally, the first frequency domain signal includes a plurality of first frequency domain frame signals, the second frequency domain signal includes a plurality of second frequency domain frame signals, and the calculating the prior probability of the frequency domain signal according to the instantaneous phase difference includes:
[0016] Obtaining a standard phase difference, and comparing the standard phase difference with the instantaneous phase difference to determine a speech indication function;
[0017] Acquire a previous frame smoothness indicator function corresponding to a previous frequency domain frame signal of each of the first frequency domain frame signals and a previous frequency domain frame signal of each of the second frequency domain frame signals;
[0018] Determine a target frame smoothness indication function corresponding to each of the first frequency domain frame signals and each of the second frequency domain frame signals by using the speech indication function and the previous frame smoothness indication function;
[0019] The prior probability is calculated according to the target frame smoothness indication function.
[0020] Optionally, before calculating the posterior probability of the presence of speech in the audio signal according to the prior probability and the synthesized signal, the method further includes:
[0021] Determining a posterior beam range of the audio signal in space according to a preset steering vector, the first frequency domain signal, and the second frequency domain signal;
[0022] Calculating the posterior probability of speech existence in the audio signal according to the prior probability and the synthesized signal comprises:
[0023] The a posteriori probability of speech existence in the audio signal is calculated according to the prior probability, the a posteriori beam range and the synthesized signal.
[0024] Optionally, determining the a posteriori beam range of the audio signal in space according to the preset steering vector, the first frequency domain signal, and the second frequency domain signal includes:
[0025] Determine angle data of the first frequency domain signal and the second frequency domain signal in different angle ranges;
[0026] For each of the angle ranges, regional energy is calculated based on the corresponding angle data and the steering vector;
[0027] The angle range with the maximum regional energy is taken as the a posteriori beam range.
[0028] Optionally, the a posteriori probability includes a first a posteriori probability and a second a posteriori probability; and calculating the a posteriori probability of speech existence in the audio signal according to the a priori probability, the a posteriori beam range and the synthesized signal includes:
[0029] Calculate a posteriori signal-to-noise ratio minimum value and noise power spectrum according to the synthetic signal;
[0030] Calculate a first a posteriori probability of speech loss according to the minimum a posteriori signal-to-noise ratio, the priori probability and the a posteriori beam range;
[0031] Calculate a posterior signal-to-noise ratio and a priori signal-to-noise ratio according to the noise power spectrum and the synthetic signal;
[0032] A second a posteriori probability of the existence of speech is calculated according to the first a posteriori probability, the a posteriori signal-to-noise ratio, and the a priori signal-to-noise ratio.
[0033] Optionally, calculating the speech spectrum gain according to the posterior probability includes:
[0034] Obtaining a general gain according to the minimum value of the a posteriori signal-to-noise ratio and the a priori signal-to-noise ratio;
[0035] Get the minimum gain;
[0036] The speech spectrum gain is calculated according to the minimum gain, the general gain and the posterior probability.
[0037] Optionally, generating a target audio signal according to the speech spectrum gain and the synthesized signal includes:
[0038] Multiplying the speech spectrum gain and the synthesized signal to obtain an output spectrum;
[0039] Performing an inverse Fourier transform on the output spectrum to obtain the target audio signal.
[0040] To achieve the above object, the present invention further provides a voice pickup device, the voice pickup device comprising:
[0041] A first acquisition module, used to acquire a first audio signal and a second audio signal, and convert the first audio signal into a first frequency domain signal, and convert the second audio signal into a second frequency domain signal; wherein the first audio signal is acquired by a first microphone, and the second audio signal is acquired by a second microphone;
[0042] A first calculation module, used for calculating the instantaneous phase difference between the first frequency domain signal and the second frequency domain signal;
[0043] A second calculation module, used for calculating the prior probability of the existence of speech in the audio signal according to the instantaneous phase difference;
[0044] A first synthesis module, used for synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal;
[0045] A third calculation module, used for calculating the posterior probability of the existence of speech in the audio signal according to the prior probability and the synthesized signal;
[0046] A fourth calculation module, used for calculating the speech spectrum gain according to the posterior probability;
[0047] The first generating module is used to generate a target audio signal according to the speech spectrum gain and the synthesized signal.
[0048] To achieve the above objectives, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the voice pickup method described above when executed by the processor.
[0049] To achieve the above object, the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the voice pickup method described above are implemented.
[0050] The present invention proposes a method, device, electronic device and computer-readable storage medium for voice pickup, which obtains a first audio signal and a second audio signal, and converts the first audio signal into a first frequency domain signal, and converts the second audio signal into a second frequency domain signal; wherein the first audio signal is collected by a first microphone, and the second audio signal is collected by a second microphone; the instantaneous phase difference between the first frequency domain signal and the second frequency domain signal is calculated; the prior probability of the existence of voice in the audio signal is calculated according to the instantaneous phase difference; the first frequency domain signal and the second frequency domain signal are synthesized to obtain a synthetic signal; the posterior probability of the existence of voice in the audio signal is calculated according to the prior probability and the synthetic signal; the voice spectrum gain is calculated according to the posterior probability; and the target audio signal is generated according to the voice spectrum gain and the synthetic signal. The prior probability is calculated by the instantaneous phase difference between the first microphone and the second microphone, so that the probability of the existence of voice can be accurately determined based on the voice pickup direction, and then the posterior probability obtained based on the prior probability can calculate the voice spectrum gain based on the actual voice scene, thereby suppressing environmental interference and improving the voice pickup performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0053] Figure 1 A schematic diagram of a flow chart of a first embodiment of a voice pickup method of the present invention;
[0054] Figure 2 is an overall flow chart of the voice pickup method of the present invention;
[0055] Figure 3 A schematic diagram of an audio signal to which the method of the present application is applied;
[0056] Figure 4 It is a schematic diagram of the module structure of the electronic device of the present invention. DETAILED DESCRIPTION
[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.
[0058] The present invention provides a voice pickup method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a voice pickup method of the present invention, wherein the method comprises the following steps:
[0059] Step S10, acquiring a first audio signal and a second audio signal, and converting the first audio signal into a first frequency domain signal, and converting the second audio signal into a second frequency domain signal; wherein the first audio signal is collected by a first microphone, and the second audio signal is collected by a second microphone;
[0060] In this embodiment, an array is formed by two microphones; the first microphone and the second microphone respectively collect sound to obtain audio signals, the audio signal collected by the first microphone is the first audio signal, and the audio signal collected by the second microphone is the second audio signal.
[0061] It can be understood that the audio signal collected by the microphone is a time domain signal, and when processing the audio signal, the audio signal needs to be converted into a frequency domain signal; specifically, the corresponding frequency domain signal can be obtained by performing a fast Fourier transform on the audio signal; the first audio signal is subjected to a fast Fourier transform to obtain a first frequency domain signal, and the second audio signal is subjected to a fast Fourier transform to obtain a second frequency domain signal.
[0062] Step S20, calculating the instantaneous phase difference between the first frequency domain signal and the second frequency domain signal;
[0063] The instantaneous phase difference is used to indicate the phase difference between the audio signals collected by the first microphone and the second microphone. It is understandable that for a single sound source, the time it takes to reach different microphones is different, resulting in a phase difference.
[0064] It can be understood that the phase difference will be reflected in the collected audio signal.
[0065] Step S30, calculating the prior probability of the presence of speech in the audio signal according to the instantaneous phase difference;
[0066] After determining the instantaneous phase difference, the prior probability of the frequency domain signal is calculated using the instantaneous phase difference. It can be understood that, in theory, except for the phase difference, the specific content of the first frequency domain signal and the second frequency domain signal is the same. Therefore, the prior probability of the existence of speech can be determined based on the phase difference. The prior probability can be used to know the relationship between the incident angle and frequency of the sound, thereby determining the probability of the existence of speech.
[0067] Step S40, synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal;
[0068] The two frequency domain signals are synthesized to obtain a synthesized signal; specifically, the synthesis can be performed by a spectrum weighting method, such as:
[0069] X ave (k) = α left ×X1(k)+α right ×X2(k)
[0070] Among them, X ave (k) is the synthesized signal, X1(k) is the first frequency domain signal, X2(k) is the second frequency domain signal, α left With α right is the weighting coefficient; k is the frequency domain subscript, 0≤k≤256.
[0071] Step S50, calculating the posterior probability of the presence of speech in the audio signal based on the prior probability and the synthesized signal;
[0072] Step S60, calculating the speech spectrum gain according to the posterior probability;
[0073] After obtaining the synthesized signal, the posterior probability of the presence or absence of speech in the synthesized signal is comprehensively calculated in combination with the prior probability; the direction of arrival of the sound can be determined through the posterior probability; thereby clarifying the direction of the effective speech and suppressing other noises; therefore, the speech spectrum gain can be determined through the posterior probability.
[0074] Step S70: generating a target audio signal according to the speech spectrum gain and the synthesized signal.
[0075] By applying the speech spectrum gain to the synthesized signal, the target audio signal with environmental interference eliminated can be obtained, thereby improving the sound pickup performance.
[0076] This embodiment calculates the prior probability through the instantaneous phase difference between the first microphone and the second microphone, so that the probability of the existence of speech can be accurately determined based on the speech pickup direction, and then the posterior probability obtained based on the prior probability can calculate the speech spectrum gain based on the actual speech scene, thereby suppressing environmental interference and improving the sound pickup performance.
[0077] Further, in the second embodiment of the voice pickup method of the present invention proposed based on the first embodiment of the present invention, the step S20 includes the steps of:
[0078] Step S21, calculating a first instantaneous phase angle of the first microphone according to the first frequency domain signal, and calculating a second instantaneous phase angle of the second microphone according to the second frequency domain signal;
[0079] Step S22, calculating the difference between the first instantaneous phase angle and the second instantaneous phase angle to obtain the instantaneous phase difference.
[0080] The first instantaneous phase angle is used to indicate the phase of the first microphone, and the second instantaneous phase angle is used to indicate the phase of the second microphone; specifically, the first instantaneous phase angle is:
[0081]
[0082] Among them, ph1(k) is the first instantaneous phase angle;
[0083] The second instantaneous phase angle is:
[0084]
[0085] Wherein, ph2(k) is the second instantaneous phase angle.
[0086] Subtracting the first instantaneous phase angle from the second instantaneous phase angle yields the instantaneous phase difference:
[0087]
[0088] in, is the instantaneous phase difference.
[0089] Further, the first frequency domain signal includes a plurality of first frequency domain frame signals, and the second frequency domain signal includes a plurality of second frequency domain frame signals. In a third embodiment of the voice pickup method of the present invention proposed based on the first embodiment of the present invention, the step S30 includes the steps of:
[0090] Step S31, obtaining a standard phase difference, and comparing the standard phase difference with the instantaneous phase difference to determine a speech indication function;
[0091] Step S32, obtaining a previous frame smoothness indicator function corresponding to a previous frequency domain frame signal of each of the first frequency domain frame signals and a previous frequency domain frame signal of each of the second frequency domain frame signals;
[0092] Step S33, determining a target frame smoothness indicator function corresponding to each of the first frequency domain frame signals and each of the second frequency domain frame signals by using the speech indicator function and the previous frame smoothness indicator function;
[0093] Step S34: calculating the prior probability according to the target frame smoothness indicator function.
[0094] The standard phase difference is a preset expected phase difference between the first microphone and the second microphone; specifically, the standard phase difference is:
[0095]
[0096] Where ζ0(k) is the standard phase difference; d is the distance between the first microphone and the second microphone; θ is the equivalent angular efficiency; c is the speed of sound, which can be 343 m / s; f s is the sampling rate, which can be 16000Hz.
[0097] The voice indication function is used to indicate the correlation between the standard phase difference and the instantaneous phase difference. Specifically, the voice indication function is:
[0098]
[0099] Where I(k) is the speech indication function.
[0100] The smoothness indicator function indicates the smooth transition between two frames of speech. Specifically, the smoothness indicator function is:
[0101]
[0102] in, is the target frame smoothness indicator function; is the smoothness indicator function of the previous frame, α is the smoothing factor, and l is the frame number.
[0103] After obtaining the smooth indicator function, the prior probability can be calculated, specifically:
[0104]
[0105] Among them, ρ(k, l) is the prior probability; k max is the maximum frequency; k min is the minimum frequency; B min is the bias compensation factor, which can be taken as 1.66.
[0106] This embodiment can accurately calculate the prior probability.
[0107] Further, in a fourth embodiment of the voice pickup method of the present invention proposed based on the first embodiment of the present invention, the method before step S50 includes:
[0108] Step S80, determining a posterior beam range of the audio signal in space according to a preset steering vector, the first frequency domain signal, and the second frequency domain signal;
[0109] The step S50 comprises:
[0110] Step S51, calculating the a posteriori probability of the presence of speech in the audio signal according to the prior probability, the a posteriori beam range and the synthesized signal.
[0111] The steering vector indicates the direction of propagation in space and the response of the array. The a posteriori beamwidth indicates where the target sound is located.
[0112] It can be understood that the relationship between the incident angle and frequency of the sound can be known through the prior probability, and the posterior beam range indicates the angular range of the target sound. Therefore, the posterior probability can be accurately determined through the prior probability and the posterior beam range.
[0113] Furthermore, the step S80 includes the steps of:
[0114] Step S81, determining angle data of the first frequency domain signal and the second frequency domain signal in different angle ranges;
[0115] Step S82, for each of the angle ranges, calculating the regional energy according to the corresponding angle data and the steering vector;
[0116] Step S83: taking the angle range with the maximum regional energy as the a posteriori beam range.
[0117] In this embodiment, the horizontal space is divided into angles, such as 60° as the dividing angle, and every 60° as an angle range, so that six angle ranges of 0-60°, 60°-120°, 120°-180°, 180°-240°, 240°-300°, and 300°-360° can be obtained; it can be understood that the sound generated by the sound source is transmitted to the microphone, and the microphone will be affected by sounds from all directions at the same time. Therefore, it is necessary to determine the direction of the target sound; the relevant signal collected within the corresponding angle range is the angle data.
[0118] Specifically, the steering vector can be calculated by time delay:
[0119] H(k,i)=exp(-jΩ(k)×τ(i))
[0120] Where i is the microphone number;
[0121]
[0122] After the steering vector is determined, the steering vector is multiplied by each angle data to obtain the regional energy of the corresponding angle range. Specifically:
[0123]
[0124] Among them, E n is the regional energy; n is the angle range number, such as 0-60° is number 1, 60°-120° is number 2, and so on.
[0125] It can be understood that the regional energy corresponds to the sound intensity of the angle range. In practical applications, the sound intensity corresponding to the target sound is the largest, so its corresponding regional energy is also the largest. Therefore, the angle range with the largest regional energy is used as the posterior beam range. Specifically:
[0126]
[0127] Among them, θ doa is the a posteriori beam range; the a posteriori beam range indicates the angular range where the target sound is located.
[0128] It can be understood that the relationship between the incident angle and frequency of the sound can be known through the prior probability, and the posterior beam range indicates the angular range of the target sound. Therefore, the posterior probability can be accurately determined through the prior probability and the posterior beam range.
[0129] Further, the posterior probability includes a first posterior probability and a second posterior probability; the step S54 includes the steps of:
[0130] Step S511, calculating a posteriori signal-to-noise ratio minimum value and noise power spectrum according to the synthetic signal;
[0131] Step S512, calculating a first a posteriori probability of speech loss according to the minimum a posteriori signal-to-noise ratio, the priori probability and the a posteriori beam range;
[0132] Step S513, calculating a posteriori signal-to-noise ratio and a priori signal-to-noise ratio according to the noise power spectrum and the synthetic signal;
[0133] Step S514: Calculate a second a posteriori probability of the presence of speech according to the first a posteriori probability, the a posteriori signal-to-noise ratio, and the a priori signal-to-noise ratio.
[0134] The first a posteriori probability indicates the probability of speech absence; the second a posteriori probability indicates the probability of speech presence.
[0135] Minimum value of the a posteriori signal-to-noise ratio: The minimum value of the a posteriori signal-to-noise ratio of the target frame; specifically:
[0136]
[0137] Among them, γ min (k, l) is the minimum value of the posterior signal-to-noise ratio; X ave (k, l) is the synthetic signal of the current frame; S min (k, l) has:
[0138] S min (k,l)=min(S min (k,l-1),|X ave (k,l)| 2 )
[0139] The noise power spectrum is the variation of noise power with frequency; specifically:
[0140] λ d (k,l)=α d λ d (k,l-1)+(1-α d )|X ave (k,l)| 2
[0141] Among them, λ d (k, l) is the target frame noise power spectrum; λ d (k, l-1) is the noise power spectrum of the previous frame; α d 0.85 is acceptable.
[0142] The first posterior probability is:
[0143]
[0144] Among them, ρ0 can be 0.65, θ0 can be 2, and γ1 can be 3.5.
[0145] The a posteriori signal-to-noise ratio is:
[0146]
[0147] The a priori signal-to-noise ratio is:
[0148]
[0149] The second posterior probability is:
[0150]
[0151] Furthermore, the step S60 includes the steps of:
[0152] Step S61, obtaining a general gain according to the minimum value of the a posteriori signal-to-noise ratio and the a priori signal-to-noise ratio;
[0153] Step S62, obtaining the minimum gain;
[0154] Step S63: Calculate the speech spectrum gain according to the minimum gain, the general gain and the posterior probability.
[0155] The general gain indicates the preliminary expected gain of the target frame; the general gain is:
[0156]
[0157] Among them, v(k, l) is:
[0158]
[0159] The minimum gain is the minimum value of the gain. The smaller the value, the stronger the suppression of noise and speech outside the wave direction, but the greater the speech distortion in the wave direction. The specific value of the minimum gain can be set based on actual needs, such as the minimum gain G0(k, l) = 10 -18 / 20 .
[0160] After the minimum gain and the general gain are determined, the final speech spectrum gain is obtained by integrating the posterior probability;
[0161] G(k,l)=G1(k,l) p(k,l) ×G0(k,l) 1-p(k,l)
[0162] Wherein, G(k, l) is the speech spectrum gain.
[0163] Further, in a fifth embodiment of the voice pickup method of the present invention proposed based on the first embodiment of the present invention, step S70 includes the steps of:
[0164] Step S71, multiplying the speech spectrum gain by the synthesized signal to obtain an output spectrum;
[0165] Step S72: performing an inverse Fourier transform on the output spectrum to obtain the target audio signal.
[0166] The output spectrum is:
[0167] Y out (k,l)=G(k,l)×X ave (k,l)
[0168] Among them, Y out (k, l) is the output spectrum; the output spectrum is the frequency domain signal corresponding to the target audio signal. It can be understood that the final target audio signal is a time domain signal. Therefore, by performing an inverse Fourier transform on the output spectrum, the output spectrum can be converted into a time domain signal to obtain the target audio signal.
[0169] Below, based on Figure 2 The overall process of this application is described as follows:
[0170] 1. The first audio signal mic1 and the second audio signal mic2 of the first microphone and the second microphone are converted into the frequency domain through time-frequency transformation to obtain a first frequency domain signal and a second frequency domain signal; fast Fourier transform can be used here;
[0171] 2. Performing spectral weighting on the first frequency domain signal and the second frequency domain signal to obtain a composite signal;
[0172] 3. Obtain the a posteriori beam range θ through the first frequency domain signal and the second frequency domain signal doa ;
[0173] 4. Calculate the prior probability of the voice pickup angle range through the first frequency domain signal and the second frequency domain signal;
[0174] 5. Obtain the noise power spectrum and the posterior probability of speech through the posterior beam range and prior probability;
[0175] 6. Calculate the speech spectrum gain based on the speech posterior probability;
[0176] 7. The synthesized signal and the speech spectrum gain module are subjected to gain transformation and time-frequency inverse transformation to obtain the output target audio signal.
[0177] See also Figure 3 , Figure 3 A schematic diagram of an audio signal to which the method of the present application is applied. Figure 3 A is the first audio signal and the second audio signal collected by the microphone, Figure 3 B is the target audio signal to be output; Figure 3 It can be seen that compared with the audio signals collected by the two microphones, the output target audio signal retains the desired signal and suppresses the undesired signal.
[0178] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.
[0179] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.
[0180] The present application also provides a voice pickup device for implementing the above-mentioned voice pickup method, the voice pickup device comprising:
[0181] A first acquisition module, used to acquire a first audio signal and a second audio signal, and convert the first audio signal into a first frequency domain signal, and convert the second audio signal into a second frequency domain signal; wherein the first audio signal is acquired by a first microphone, and the second audio signal is acquired by a second microphone;
[0182] A first calculation module, used for calculating the instantaneous phase difference between the first frequency domain signal and the second frequency domain signal;
[0183] A second calculation module, used for calculating the prior probability of the existence of speech in the audio signal according to the instantaneous phase difference;
[0184] A first synthesis module, used for synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal;
[0185] A third calculation module, used for calculating the posterior probability of the existence of speech in the audio signal according to the prior probability and the synthesized signal;
[0186] A fourth calculation module, used for calculating the speech spectrum gain according to the posterior probability;
[0187] The first generating module is used to generate a target audio signal according to the speech spectrum gain and the synthesized signal.
[0188] The voice pickup device calculates the prior probability through the instantaneous phase difference between the first microphone and the second microphone, so that the probability of the existence of the voice can be accurately determined based on the voice pickup direction, and then the posterior probability obtained based on the prior probability can calculate the voice spectrum gain based on the actual voice scene, thereby suppressing environmental interference and improving the sound pickup performance.
[0189] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the first calculation module in this embodiment can be used to execute step S20 in the embodiment of the present application, the second calculation module in this embodiment can be used to execute step S30 in the embodiment of the present application, the first synthesis module in this embodiment can be used to execute step S40 in the embodiment of the present application, the third calculation module in this embodiment can be used to execute step S50 in the embodiment of the present application, the fourth calculation module in this embodiment can be used to execute step S60 in the embodiment of the present application, and the first generation module in this embodiment can be used to execute step S70 in the embodiment of the present application.
[0190] Furthermore, the first calculation module includes:
[0191] a first calculation unit, configured to calculate a first instantaneous phase angle of the first microphone according to the first frequency domain signal, and calculate a second instantaneous phase angle of the second microphone according to the second frequency domain signal;
[0192] The second calculation unit is used to calculate the difference between the first instantaneous phase angle and the second instantaneous phase angle to obtain the instantaneous phase difference.
[0193] Further, the first frequency domain signal includes a plurality of first frequency domain frame signals, the second frequency domain signal includes a plurality of second frequency domain frame signals, and the second calculation module includes:
[0194] A first acquisition unit, configured to acquire a standard phase difference, and compare the standard phase difference with the instantaneous phase difference to determine a speech indication function;
[0195] A second acquisition unit, configured to acquire a previous frame smoothness indicator function corresponding to a previous frequency domain frame signal of each of the first frequency domain frame signals and a previous frequency domain frame signal of each of the second frequency domain frame signals;
[0196] A first determining unit, configured to determine a target frame smoothness indication function corresponding to each of the first frequency domain frame signals and each of the second frequency domain frame signals by using the speech indication function and the previous frame smoothness indication function;
[0197] The third calculation unit is used to calculate the prior probability according to the target frame smoothness indication function.
[0198] Furthermore, the device also includes:
[0199] A first determination module, configured to determine a posterior beam range of an audio signal in space according to a preset steering vector, the first frequency domain signal, and the second frequency domain signal;
[0200] The third calculation module includes:
[0201] The fourth calculation unit is used to calculate the posterior probability of the existence of speech in the audio signal according to the prior probability, the posterior beam range and the synthesized signal.
[0202] Furthermore, the first determining module includes:
[0203] A third acquisition unit, used to determine angle data of the first frequency domain signal and the second frequency domain signal in different angle ranges;
[0204] a fifth calculation unit, configured to calculate, for each angle range, regional energy according to the corresponding angle data and the steering vector;
[0205] The first execution unit is used to use the angle range with the maximum regional energy as the a posteriori beam range.
[0206] Further, the posterior probability includes a first posterior probability and a second posterior probability; and the fourth calculation unit includes:
[0207] A first calculation subunit, used for calculating a posteriori signal-to-noise ratio minimum value and a noise power spectrum according to the synthetic signal;
[0208] A second calculation subunit, configured to calculate a first a posteriori probability of speech missing according to the minimum a posteriori signal-to-noise ratio, the priori probability and the a posteriori beam range;
[0209] A third calculation subunit, configured to calculate a posterior signal-to-noise ratio and a priori signal-to-noise ratio according to the noise power spectrum and the synthetic signal;
[0210] The fourth calculation subunit is used to calculate a second a posteriori probability of the existence of speech according to the first a posteriori probability, the a posteriori signal-to-noise ratio, and the a priori signal-to-noise ratio.
[0211] Furthermore, the fourth calculation module includes:
[0212] A second execution unit, configured to obtain a general gain according to the minimum value of the a posteriori signal-to-noise ratio and the a priori signal-to-noise ratio;
[0213] A fourth acquisition unit, used to acquire a minimum gain;
[0214] The sixth calculation unit is used to calculate the speech spectrum gain according to the minimum gain, the general gain and the posterior probability.
[0215] Furthermore, the first generating module includes:
[0216] a seventh calculation unit, configured to multiply the speech spectrum gain by the synthesized signal to obtain an output spectrum;
[0217] An eighth calculation unit is used to perform an inverse Fourier transform on the output spectrum to obtain the target audio signal.
[0218] Reference Figure 4 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.
[0219] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other electronic devices, servers or Internet of Things devices, such as televisions, etc.
[0220] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as obtaining a first audio signal and a second audio signal), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0221] The processor 30 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the electronic device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.
[0222] although Figure 4 Although not shown, the electronic device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 4 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0223] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 4 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0224] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0225] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0226] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.
Claims
1. A voice pickup method, characterized in that: The voice pickup method comprises: Acquire a first audio signal and a second audio signal, and convert the first audio signal into a first frequency domain signal, and convert the second audio signal into a second frequency domain signal; wherein the first audio signal is collected by a first microphone, and the second audio signal is collected by a second microphone; Calculating an instantaneous phase difference between the first frequency domain signal and the second frequency domain signal; Calculating a priori probability of speech presence in the audio signal based on the instantaneous phase difference; synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal; Calculate the posterior probability of the presence of speech in the audio signal based on the prior probability and the synthesized signal; Calculate the speech spectrum gain according to the posterior probability; A target audio signal is generated according to the speech spectrum gain and the synthesized signal.
2. The voice pickup method according to claim 1, characterized in that: The calculating the instantaneous phase difference between the first microphone and the second microphone according to the first frequency domain signal and the second frequency domain signal comprises: Calculate a first instantaneous phase angle of the first microphone according to the first frequency domain signal, and calculate a second instantaneous phase angle of the second microphone according to the second frequency domain signal; The difference between the first instantaneous phase angle and the second instantaneous phase angle is calculated to obtain the instantaneous phase difference.
3. The voice pickup method according to claim 1, characterized in that: The first frequency domain signal includes a plurality of first frequency domain frame signals, the second frequency domain signal includes a plurality of second frequency domain frame signals, and the calculating the prior probability of the frequency domain signal according to the instantaneous phase difference includes: Obtaining a standard phase difference, and comparing the standard phase difference with the instantaneous phase difference to determine a speech indication function; Acquire a previous frame smoothness indicator function corresponding to a previous frequency domain frame signal of each of the first frequency domain frame signals and a previous frequency domain frame signal of each of the second frequency domain frame signals; Determine a target frame smoothness indication function corresponding to each of the first frequency domain frame signals and each of the second frequency domain frame signals by using the speech indication function and the previous frame smoothness indication function; The prior probability is calculated according to the target frame smoothness indication function.
4. The voice pickup method according to claim 1, characterized in that: Before calculating the posterior probability of the presence of speech in the audio signal according to the prior probability and the synthesized signal, the method further comprises: Determining a posterior beam range of the audio signal in space according to a preset steering vector, the first frequency domain signal, and the second frequency domain signal; Calculating the posterior probability of speech existence in the audio signal according to the prior probability and the synthesized signal comprises: The a posteriori probability of speech existence in the audio signal is calculated according to the prior probability, the a posteriori beam range and the synthesized signal.
5. The voice pickup method according to claim 4, characterized in that: The step of determining the a posteriori beam range of the audio signal in space according to the preset steering vector, the first frequency domain signal, and the second frequency domain signal comprises: Determine angle data of the first frequency domain signal and the second frequency domain signal in different angle ranges; For each of the angle ranges, regional energy is calculated based on the corresponding angle data and the steering vector; The angle range with the maximum regional energy is taken as the a posteriori beam range.
6. The voice pickup method according to claim 4, characterized in that: The a posteriori probability includes a first a posteriori probability and a second a posteriori probability; and the a posteriori probability of the presence of speech in the audio signal calculated according to the a priori probability, the a posteriori beam range and the synthesized signal includes: Calculate a posteriori signal-to-noise ratio minimum value and noise power spectrum according to the synthetic signal; Calculate a first a posteriori probability of speech loss according to the minimum a posteriori signal-to-noise ratio, the priori probability and the a posteriori beam range; Calculate a posterior signal-to-noise ratio and a priori signal-to-noise ratio according to the noise power spectrum and the synthetic signal; A second a posteriori probability of the existence of speech is calculated according to the first a posteriori probability, the a posteriori signal-to-noise ratio, and the a priori signal-to-noise ratio.
7. The voice pickup method according to claim 6, characterized in that: Calculating the speech spectrum gain according to the posterior probability comprises: Obtaining a general gain according to the minimum value of the a posteriori signal-to-noise ratio and the a priori signal-to-noise ratio; Get the minimum gain; The speech spectrum gain is calculated according to the minimum gain, the general gain and the posterior probability.
8. The voice pickup method according to claim 1, characterized in that: Generating a target audio signal according to the speech spectrum gain and the synthesized signal comprises: Multiplying the speech spectrum gain and the synthesized signal to obtain an output spectrum; Performing an inverse Fourier transform on the output spectrum to obtain the target audio signal.
9. A voice pickup device, characterized in that: The voice pickup device comprises: A first acquisition module, used to acquire a first audio signal and a second audio signal, and convert the first audio signal into a first frequency domain signal, and convert the second audio signal into a second frequency domain signal; wherein the first audio signal is acquired by a first microphone, and the second audio signal is acquired by a second microphone; A first calculation module, used for calculating the instantaneous phase difference between the first frequency domain signal and the second frequency domain signal; A second calculation module, used for calculating the prior probability of the existence of speech in the audio signal according to the instantaneous phase difference; A first synthesis module, used for synthesizing the first frequency domain signal and the second frequency domain signal to obtain a synthesized signal; A third calculation module, used for calculating the posterior probability of the existence of speech in the audio signal according to the prior probability and the synthesized signal; A fourth calculation module, used for calculating the speech spectrum gain according to the posterior probability; The first generating module is used to generate a target audio signal according to the speech spectrum gain and the synthesized signal.
10. An electronic device, characterized in that: The electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the voice pickup method according to any one of claims 1 to 8 when executed by the processor.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the voice pickup method according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Single-microphone long-distance pickup method and device
CN120321534A
