Array-based sound pickup method and device having continuously variable polar patterns

Abstract

The present application discloses an array-based sound pickup method and device having continuously variable polar patterns. The method comprises: receiving a first audio from a unidirectional microphone and a second audio from an omnidirectional microphone, and calculating a gain coefficient; performing gain processing on the first audio on the basis of the gain coefficient to obtain a first gain-processed audio; performing delay alignment on the second audio to obtain an auxiliary audio; mixing the first gain-processed audio and the auxiliary audio according to a proportional value to obtain a mixed audio; and performing noise reduction on the mixed audio by means of a noise reduction strategy and the gain coefficient to obtain a corresponding target audio. The above method is used in conjunction with an array-based sound pickup device capable of continuously variable polar pattern adjustment; two or more microphones having different polar patterns are used to form a sound pickup array, and a specific microphone array algorithm is used to achieve wide-range continuously variable adjustment of pickup directivity; moreover, by means of an array-based post-processing noise reduction algorithm, a stronger noise reduction function than a conventional single microphone or simple array-based sound pickup is achieved.

Description

A method and apparatus for array pickup with stepless adjustment of pickup directionality

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411704321.X, filed on November 26, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of audio processing technology, and in particular to an array pickup method and apparatus with infinitely adjustable pickup direction. Background Technology

[0003] To meet the application requirements of different microphone pickup patterns, some professional microphones can change their pickup pattern through external circuitry (such as switching to cardioid or bidirectional pickup patterns). Other microphone devices utilize circuitry to switch between different pickup patterns to achieve a change in the overall output pickup pattern. In general, all existing methods and devices for changing microphone pickup patterns utilize some form of switching (hardware or software) to select different pickup devices, thereby changing the microphone's output direction. Its disadvantages are: 1. The microphone's output directionality is limited by the directivity of the original pickup device. Considering the overall size of the microphone, it is generally impossible to integrate more microphones with different directions to achieve multiple directional outputs; 2. Its output directionality cannot exceed the directional range of the original pickup device: for example, a combination of a cardioid and an omnidirectional microphone can only output cardioid or omnidirectional, and cannot achieve supercardioid or bidirectional directional; 3. Its directional change is a discrete switch, and it cannot achieve a gradual change in directionality, making it difficult to meet the usage scenarios that require fine adjustment of the microphone's directionality; 4. This type of microphone only uses one pickup device at a time, and does not effectively utilize the array pickup characteristics formed by multiple microphones, resulting in poor post-processing noise reduction performance.

[0004] Another type of microphone array based on differential arrays can adjust the pickup directionality of the array by adjusting the delay differential mixing ratio of different array elements. However, it generally uses array elements with uniform directionality (usually omnidirectional microphones) to implement the array. Its disadvantage is that it is difficult to achieve strong directionality when the number of array elements is small. Therefore, microphone arrays used for pickup in existing technologies suffer from poor pickup directionality.

[0005] Application content

[0006] This application provides a microphone array pickup method and apparatus with stepless adjustable pickup direction, aiming to solve the problem of poor pickup directionality of microphone arrays used for pickup in existing technologies.

[0007] In a first aspect, embodiments of this application provide a method for array microphones with infinitely adjustable pickup directionality, wherein the method is applied in a microphone controller, the microphone controller being communicatively connected to an adjustable pickup array, the adjustable pickup array consisting of a unidirectional microphone and a non-directional microphone, and the method comprising:

[0008] If a first audio signal is received from the unidirectional microphone and a second audio signal is received from the non-directional microphone, the gain coefficients corresponding to the first audio signal and the second audio signal are calculated according to a preset audio gain strategy.

[0009] The first audio is subjected to gain processing based on the gain coefficient to obtain the corresponding first gain audio;

[0010] The second audio is time-aligned according to the preset delay duration to obtain the corresponding auxiliary audio;

[0011] The first gain audio and the auxiliary audio are mixed according to a preset ratio value to obtain the corresponding mixed audio.

[0012] The mixed audio is denoised according to the preset noise reduction strategy and the gain coefficient to obtain the corresponding target audio.

[0013] Secondly, embodiments of this application also provide an array pickup device with stepless adjustable pickup direction, wherein the device includes a device body, a pickup controller and a pickup array with adjustable directionality, and the pickup controller is used to execute the array pickup method with stepless adjustable pickup direction as described in the first aspect above.

[0014] The microphone controller is disposed within the device body;

[0015] The directional adjustable pickup array consists of a unidirectional microphone and a non-directional microphone; both the unidirectional microphone and the non-directional microphone are disposed on the same outer surface of the device body; the pickup direction of the unidirectional microphone is opposite to that of the non-directional microphone.

[0016] The pickup controller is communicatively connected to both the unidirectional microphone and the nondirectional microphone.

[0017] This application provides a method and apparatus for array microphone pickup with stepless adjustment of pickup directionality. The method includes: receiving a first audio signal from a unidirectional microphone and a second audio signal from an omnidirectional microphone and calculating a gain coefficient; performing gain processing on the first audio signal according to the gain coefficient to obtain a first gain audio signal; performing delay alignment on the second audio signal to obtain an auxiliary audio signal; performing mixing processing on the first gain audio signal and the auxiliary audio signal according to a ratio value to obtain a mixed audio signal; and performing noise reduction processing on the mixed audio signal according to a noise reduction strategy and the gain coefficient to obtain a corresponding target audio signal. This method, combined with an array microphone pickup apparatus that can steplessly adjust the pickup directionality, utilizes two or more microphones with different pickup directions to form a pickup array and employs a specific microphone array algorithm to achieve a wide range of stepless adjustment of pickup directionality. Furthermore, combined with an array post-processing noise reduction algorithm, it achieves stronger noise reduction capabilities than traditional single-microphone or simple array microphone pickup. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 is a flowchart of the array pickup method with stepless adjustment of pickup direction provided in the embodiment of this application;

[0020] Figure 2 is a schematic diagram of the application scenario of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0021] Figure 3 is a schematic diagram of the audio processing flow of the array pickup method with stepless adjustable pickup direction provided in the embodiment of this application;

[0022] Figure 4 is a schematic diagram of a partial audio processing flow of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0023] Figure 5 is a schematic diagram of the effect of the array pickup method with stepless adjustable pickup direction provided in the embodiment of this application;

[0024] Figure 6 is a schematic diagram of another audio processing flow of the array pickup method with stepless adjustable pickup direction provided in the embodiment of this application;

[0025] Figure 7 is another flowchart of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0026] Figure 8 is a schematic diagram of another application scenario of the array pickup method with stepless adjustable pickup direction provided in the embodiments of this application;

[0027] Figure 9 is a schematic diagram of another audio processing flow of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0028] Figure 10 is another audio processing flowchart of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0029] Figure 11 is another flowchart of the array pickup method with stepless adjustable pickup direction provided in the embodiment of this application;

[0030] Figure 12 is a schematic diagram of another application scenario of the array pickup method with stepless adjustable pickup direction provided in the embodiment of this application;

[0031] Figure 13 is a schematic diagram of the subsequent audio processing flow of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application.

[0032] Figure 14 is another flowchart of the array pickup method with stepless pickup direction adjustment provided in the embodiment of this application;

[0033] Figure 15 is a schematic block diagram of an array pickup device with stepless adjustable pickup direction provided in an embodiment of this application;

[0034] Figure 16 is another schematic block diagram of the array pickup device with stepless pickup direction adjustment provided in the embodiment of this application;

[0035] Figure 17 is another schematic block diagram of the array pickup device with stepless pickup direction adjustment provided in the embodiment of this application;

[0036] Figure 18 is another schematic block diagram of the array pickup device with stepless adjustable pickup direction provided in the embodiment of this application;

[0037] Figure 19 is a schematic block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0040] It should also be understood that the terminology used in this application specification is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this application specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0041] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0042] Please refer to Figure 1. As shown in the figure, an embodiment of this application provides a method for array microphones with infinitely adjustable pickup direction. This method is applied in a microphone controller and is executed by application software installed in the microphone controller. As shown in Figures 2 and 15, a communication connection is established between the microphone controller 10 and the directional adjustable microphone array 20. The directional adjustable microphone array 20 consists of a unidirectional microphone 21 and an omnidirectional microphone 22. The microphone controller 10 can establish wired or wireless connections with each microphone, for example, the microphone controller 10 can establish a Bluetooth / RF wireless connection with the microphone, or a wired connection through a data transmission line. As shown in Figure 1, the method includes steps S110 to S150.

[0043] S110. If a first audio signal from the unidirectional microphone and a second audio signal from the non-directional microphone are received, the gain coefficients corresponding to the first audio signal and the second audio signal are calculated according to a preset audio gain strategy.

[0044] An omnidirectional microphone is also known as a single-pointed microphone. Omnidirectional microphones and single-pointed microphones are located in the same physical pickup component. Their relative positions are fixed. The single-pointed microphone is the main pickup microphone, and its pickup direction is the main pickup direction, mainly picking up human voices in that direction. The omnidirectional microphone is the auxiliary pickup microphone, which can pick up more environmental noise. A pickup array composed of omnidirectional microphones and single-pointed microphones is shown in Figure 2.

[0045] When directional sound pickup is required, array algorithms can further enhance the sound in the main pickup direction while suppressing noise in other directions, achieving better directionality and sidelobe suppression than supercardioid microphones. After delay alignment of the array-enhanced output and the omnidirectional microphone output, they are mixed proportionally and noise-reduced to obtain the final audio output. The DSP (Digital Signal Processor) processing flowchart for the directional adjustable pickup array is shown in Figure 3. The unidirectional microphone input is x1(t), and the omnidirectional microphone input is x2(t). After A / D sampling and quantization, they become digital signal sequences x1(n) and x2(n). The digital signal sequence x1(n) is the digital signal sequence corresponding to the first audio frequency, and x2(n) is the digital signal sequence corresponding to the second audio frequency. First, array enhancement processing is performed on the digital signal sequences corresponding to the first and second audio frequencies using an audio gain strategy.

[0046] In a specific embodiment, step S110 includes the following sub-steps: calculating the first energy function and the second energy function corresponding to the first audio and the second audio respectively according to the average amplitude function in the audio gain strategy; calculating the first energy function and the second energy function according to the gain calculation formula in the audio gain strategy to obtain the gain coefficient corresponding to the current audio frame.

[0047] First, array enhancement processing can be performed on x1(n) and x2(n). In a specific embodiment, a time-domain differential array processing method based on short-time energy function can be used for array enhancement processing, and its specific processing flow is shown in Figure 4. Here, x1′(n) and x2′(n) obtained after gain calibration processing can be used for calculation. Then, x1′(n) is the digital signal sequence after gain calibration processing of x1(n) with calibration coefficients, and x2′(n) is the digital signal sequence after gain calibration processing of x2(n) with calibration coefficients.

[0048] The first energy function E1′(n) and the second energy function E2′(n) corresponding to x1′(n) and x2′(n) are calculated using the average amplitude function, respectively. The formula for calculating the short-time energy function E is shown in equation (1):

[0049] Where m is the summation variable of the data in the s-th frame, N is the frame length, and n is the sample number of the current audio frame.

[0050] Furthermore, the first and second energy functions obtained can be calculated according to the gain calculation formula to obtain the gain coefficient corresponding to the current audio frame. The gain calculation formula is shown in equation (2):

[0051] Here, Δ is a minimum value, which is used to prevent excessive output in the Mask gain calculation. Mask0 is the normalized gain value, which is the value that makes the mean value of Mask(n) 1 when the sound source at a point 0.5m in the main pickup direction is emitted.

[0052] In another embodiment, step S110 includes the following sub-steps: performing short-time Fourier transforms on the first audio and the second audio according to the transform calculation formula in the audio gain strategy to obtain corresponding first frequency domain signals and second frequency domain signals; calculating the first energy function and the second energy function corresponding to the first frequency domain signal and the second frequency domain signal respectively according to the average amplitude function in the audio gain strategy; and calculating the first energy function and the second energy function according to the gain calculation formula in the audio gain strategy to obtain the gain coefficient corresponding to the current audio frame.

[0053] In another embodiment, the audio gain strategy can also be a strategy based on the array processing algorithm corresponding to the frequency domain or sub-band. Specifically, firstly, the first audio and the second audio can be subjected to short-time Fourier transform according to the transform calculation formula, that is, x1′(n) and x2′(n) are first transformed into frequency domain signals X1′(k,n) and X2′(k,n) through STFT short-time Fourier transform, where k in parentheses represents the sub-band number; then X1′(k,n) is the first frequency domain signal corresponding to the digital signal sequence of the first audio, and X2′(k,n) is the second frequency domain signal corresponding to the numerical signal sequence of the second audio. The first energy function E1′(k,n) corresponding to X1′(k,n) is calculated using equation (1), and the second energy function E2′(k,n) corresponding to X2′(k,n) is calculated using equation (1). Then, the corresponding gain coefficient is calculated in each ERB frequency band using the gain calculation formula. The gain coefficient is also the Mask(k,n) value. The specific calculation process is shown in equation (6).

[0054] Subsequent steps all involve audio processing based on the gain coefficient Mask(k,n), which is then applied to X1(k,n) to obtain the array-enhanced frequency domain signal Y(k,n), i.e., the first-gain audio signal. Specifically, Y(k,n) = X1′(k,n) × Mask ERB(k,n). Then, it is proportionally mixed with the auxiliary audio spectrum signal X2′(k,n-τ) after delay alignment and phase calibration to obtain the directional adjustable spectrum output signal Y′(k,n). The specific process is: Y′(k,n)=k×Y(k,n)+(1-k)×X2′(k,n-τ), k∈[0,1]. At this time, Y′(k,n) is used as the mixed audio for further noise reduction processing to obtain the target audio, that is, frequency domain noise reduction processing is performed on each ERB band to obtain Z(k,n). The specific process is: Z(k,n)=F DRC (Mask ERB (k,n),n)×Y′(k,n),F DRC (Mask ERB Z(k,n) is the DRC processing function. Finally, ISTFT (Inverse Short Time Fourier Transform) is performed on Z(k,n) to obtain the final time-domain signal z(n). The obtained z(n) is also the target audio. The specific process of the above audio processing is shown in Figure 6.

[0055] The time-domain processing and frequency-domain processing methods of this application each have their own advantages and disadvantages. The time-domain processing method has almost no time delay, but its directional characteristics and noise reduction effect are slightly worse; the frequency-domain processing method has good directional characteristics and excellent noise reduction effect, but its processing time delay is slightly larger.

[0056] In a specific embodiment, before step S110, the method further includes the following steps: acquiring a first test audio and a second test audio obtained by the microphone capturing audio from a sound source at a predetermined location; the predetermined location is a position in the normal direction of the line connecting the microphone and at a predetermined distance from the line connecting the microphone; the microphone line is the line connecting the unidirectional microphone and the non-directional microphone or the line connecting the unidirectional microphone and the composite pickup microphone; and configuring calibration coefficients corresponding to the unidirectional microphone and the other microphone according to the volume of the first test audio and the second test audio, so that the average output volume of the first test audio and the second test audio after calibration by the calibration coefficients are equal.

[0057] Furthermore, since the sensitivities of omnidirectional and monodirectional microphones in the above embodiments may not be the same, gain calibration can be performed before the first and second audio frequencies are collected to improve the quality of the acquired audio. Specifically, the point source can be set to be located in the normal direction of the microphone connection line, and the distance from the microphone connection line can be d, i.e., a preset distance of d. By adjusting the calibration coefficient, the average output volume of the two omnidirectional and monodirectional microphones after calibration is made equal. That is, the two microphones are calibrated by the calibration coefficient to make the average output amplitude of the two microphones the same. Different microphones are configured with different calibration coefficients. For example, the calibration coefficient configured for the monodirectional microphone is 0.8, and the calibration coefficient configured for the omnidirectional microphone is 1.1. Then, the loudness of the audio acquired by the monodirectional microphone is multiplied by 0.8 to obtain the calibrated audio; the loudness of the audio acquired by the omnidirectional microphone is multiplied by 1.1 to obtain the calibrated audio. The value of d depends on the usage scenario of the microphone. For example, for handheld microphones, d is generally between 0.5m and 1m. After gain calibration, the microphone output signals are x1′(n) and x2′(n). In subsequent processing steps, the audio obtained after gain calibration of the audio acquired by the omnidirectional microphone can be used as the second audio, and the audio obtained after gain calibration of the audio acquired by the unidirectional microphone can be used as the first audio.

[0058] S120. Perform gain processing on the first audio according to the gain coefficient to obtain the corresponding first gain audio.

[0059] The first audio is processed by gaining the obtained gain coefficient to obtain the first gain audio, which is also the array enhancement output y(n). The specific calculation process is shown in equation (3): y(n)=x1′(n)×Mask(n) (3);

[0060] S130. The second audio is time-aligned according to the preset delay duration to obtain the corresponding auxiliary audio.

[0061] The second audio can be time-aligned according to a preset delay duration, that is, time-aligned x2′(n). The specific expression for the resulting auxiliary audio is x2′(t-τ), where t is the audio frame time of the second audio and τ is the delay duration. The value of τ in the above formula depends on the spacing and sampling rate between the unidirectional and omnidirectional microphones. The purpose is to ensure that the two microphones are in phase when picking up the sound in the main pickup direction.

[0062] S140. The first gain audio and the auxiliary audio are mixed according to a preset ratio value to obtain the corresponding mixed audio.

[0063] The first gain audio and the auxiliary audio are mixed according to the preset ratio value. The ratio value is also the mixing ratio coefficient p. The value of p ranges from [0,1]. The higher the value of p, the higher the output ratio of the array enhancement algorithm and the better the pickup directivity. The lower the value, the higher the output ratio of the omnidirectional microphone and the more sounds from other directions can be picked up. The specific process of mixing is shown in equation (4): y′(n)=p×y(n)+(1-p)×x2′(t-τ) (4);

[0064] The resulting mixed audio is y′(n); in the above formula, p is the mixing ratio. The larger the value of p, the larger the proportion in the output y(n), and thus the stronger the directionality of the output y′(n). Conversely, the weaker the directionality of the output y′(n).

[0065] S150. The mixed audio is subjected to noise reduction processing according to the preset noise reduction strategy and the gain coefficient to obtain the corresponding target audio.

[0066] Finally, the mixed audio is denoised according to the denoising strategy and the above gain coefficients, that is, y′(n) is denoised to obtain z(n), and z(n) is the target audio obtained. The purpose of the denoising is to further enhance the pickup difference between the signal in the main pickup direction and the environmental noise in other directions. In this embodiment, a nonlinear compression method similar to Dynamic Range Compression Control (DRC) is used to achieve the denoising. Unlike the traditional DRC implementation of noise gate processing, the instantaneous gain of the traditional DRC depends on the input envelope amplitude of the current signal, while the instantaneous gain in this embodiment depends on the Mask gain of the current frame. The specific denoising process is shown in Equation (5): z(n)=f DRC (Mask(n),n)×y′(n) (5);

[0067] Among them, f DRC (Mask(n), n) is the DRC processing function, and its function curve is shown in Figure 5. Mask(n) is used as the input of the function, f DRC (Mask(n), n) is the output.

[0068] Please refer to Figure 7. As shown in the figure, an embodiment of this application provides a continuously adjustable pickup array method. This method is applied in a pickup controller and is executed by application software installed in the pickup controller. As shown in Figures 8 and 16, a communication connection is established between the pickup controller 10 and the composite differential pickup array 30. The composite differential pickup array 30 consists of a composite pickup microphone 31 and an omnidirectional microphone 22. The composite pickup microphone 31 is composed of two symmetrically combined omnidirectional microphones. As shown in Figure 7, the method includes steps S210 to S260.

[0069] S210. If a first sub-audio tone and a second sub-audio tone are received from the composite microphone, and a second audio tone is received from the omnidirectional microphone, the first sub-audio tone and the second sub-audio tone are combined according to a preset directional adjustment parameter to obtain the corresponding first audio tone.

[0070] Two omnidirectional microphones are stacked together to form a composite microphone, which replaces the single-pointing microphone in the above embodiment, thus obtaining the composite differential microphone array shown in Figure 8. The time-domain processing algorithm flow is shown in Figure 9.

[0071] Since the composite microphone can acquire the first sub-audio and the second sub-audio through its two internal omnidirectional microphones, a directional adjustment parameter Pdir needs to be introduced. This directional adjustment parameter Pdir can control the directionality of the differential array output, thereby enabling the microphone to achieve a larger directional adjustment range and realize bidirectional pickup.

[0072] S220. Calculate the gain coefficients corresponding to the first audio and the second audio according to a preset audio gain strategy. S230. Perform gain processing on the first audio according to the gain coefficients to obtain the corresponding first-gain audio. S240. Perform delay alignment on the second audio according to a preset delay duration to obtain the corresponding auxiliary audio. S250. Perform mixing processing on the first-gain audio and the auxiliary audio according to a preset ratio value to obtain the corresponding mixed audio. S260. Perform noise reduction processing on the mixed audio according to a preset noise reduction strategy and the gain coefficients to obtain the corresponding target audio.

[0073] The specific process of calculating the gain coefficients corresponding to the first and second audio frequencies based on the preset audio gain strategy can also be implemented using either time-domain or frequency-domain processing methods. The frequency-domain processing method is shown in Figure 10, and the time-domain processing method is shown in Figure 9. The actual processing procedures for steps S220 to S260 are similar to those for steps S110 to S150, and will not be described in detail here.

[0074] Please refer to Figure 11. As shown in the figure, an embodiment of this application provides a method for array microphones with infinitely adjustable pickup direction. This method is applied in a microphone controller and a signal processor, and is executed by application software installed in the microphone controller and the signal processor. As shown in Figures 12 and 17, the microphone controller 10 is communicatively connected to the directional adjustable microphone array 20 and the signal processor 41, respectively. The directional adjustable microphone array 20 consists of a unidirectional microphone 21 and an omnidirectional microphone 22. The microphone controller 10 can establish a wired or wireless communication connection with the signal processor 41, and the signal processor 41 can communicate with the independent omnidirectional microphone 42 to obtain the corresponding independent pickup audio from the independent omnidirectional microphone 42. As shown in Figure 11, the method includes steps S310 to S380.

[0075] S310. If the pickup controller receives a first audio signal from the unidirectional microphone and a second audio signal from the non-directional microphone, it sends the first audio signal to the signal processor. S320. The pickup controller calculates a gain coefficient corresponding to the first audio signal and the second audio signal according to a preset audio gain strategy. S330. The pickup controller performs gain processing on the first audio signal according to the gain coefficient to obtain a corresponding first gain audio signal and sends it to the signal processor. S340. The pickup controller performs delay alignment on the second audio signal according to a preset delay duration to obtain a corresponding auxiliary audio signal. S351. The signal processor performs delay alignment on the second audio signal according to a preset blocking matrix. The first audio and the independent pickup audio acquired by the independent omnidirectional microphone are subjected to blocking filtering to obtain the corresponding blocking audio after removing the original audio in the target direction; S360, the signal processor performs adaptive filtering enhancement on the first gain audio according to a preset filtering enhancement strategy and the blocking audio to obtain the corresponding first filtered audio and filter coefficients and sends them to the pickup controller; S370, the pickup controller performs mixing processing on the first filtered audio and the auxiliary audio according to a preset ratio value to obtain the corresponding mixed audio; S380, the pickup controller performs post-filtering processing on the mixed audio according to the filter coefficients to obtain the corresponding target audio.

[0076] In this embodiment, the device corresponding to the independent omnidirectional microphone can be used as an independent pickup microphone. The independent omnidirectional microphone is designated as microphone 2, and the directional adjustable pickup array is designated as microphone 1. The distance and relative position between microphone 2 and microphone 1 are not fixed. Microphone 1 and microphone 2 can form a distributed pickup array, resulting in better noise reduction. In Figure 12, microphone 2 is generally an accessory to microphone 1 rather than a completely independent device. For example, if microphone 1 is a wireless microphone, microphone 2 may be an auxiliary pickup microphone located on the receiver of microphone 1 (i.e., microphone 2 in Figure 12). Its function is to receive more ambient sound, further enhancing the noise reduction effect in unidirectional pickup, and more evenly picking up sound from all directions in omnidirectional pickup. The DSP processing flow executed by the pickup controller and signal processor is shown in Figure 13. Compared to the embodiment that only sets a directional adjustable pickup array, an additional TF-GSC array processing stage is added. Among them, TF-GSC is also known as Transfer Function Generalized Sidelobe Canceler. The difference between it and the traditional GSC (Generalized Sidelobe Canceler) algorithm is that since the external independent omnidirectional microphone is actually set on microphone 2 (microphone 2 is the receiver of microphone 1), the TF-GSC array in Figure 13 is actually located on the signal processor that communicates with the independent omnidirectional microphone. The signal processor obtains the first gain audio of microphone 1 (that is, the array enhancement output y(k)) as the main input, obtains the independent pickup audio collected by the independent omnidirectional microphone itself as the auxiliary input, and outputs C(k) after TF-GSC processing as the first filtered audio. The first filtered audio C(k) and the auxiliary audio X(2) are mixed proportionally based on a preset ratio value, and then subjected to frequency domain post-filtering processing. The post-filtering here is not the aforementioned DRC-based noise reduction algorithm, but the classic log-MMSE post-filtering algorithm, so as to obtain the corresponding target audio. In step S350, the target direction original audio is the original audio with the same direction as the final target audio. The target direction original audio is the original audio corresponding to the target audio that has not been processed by steps S360 to S380.

[0077] Please refer to Figure 14. As shown in the figure, an embodiment of this application provides an array pickup method with stepless adjustable pickup direction. This method is applied in a pickup controller and a signal processor, and is executed by application software installed in the pickup controller and the signal processor. As shown in Figure 18, the pickup controller 10 is communicatively connected to the composite differential pickup array 30 and the signal processor 41. The composite differential pickup array 30 consists of a composite pickup microphone 31 and an omnidirectional microphone 22. The composite pickup microphone 31 is composed of two symmetrically combined omnidirectional microphones. The signal processor 41 is communicatively connected to an independent omnidirectional microphone 42 to obtain the corresponding independent pickup audio from the independent omnidirectional microphone 42. As shown in Figure 14, the method includes steps S410 to S480.

[0078] S410. If the pickup controller receives a first sub-audio and a second sub-audio from the composite pickup microphone, and a second audio from the omnidirectional microphone, it performs audio composite processing on the first sub-audio and the second sub-audio according to a preset directional adjustment parameter to obtain a corresponding first audio and sends it to the signal processor; S420. The pickup controller calculates a gain coefficient corresponding to the first audio and the second audio according to a preset audio gain strategy; S430. The pickup controller performs gain processing on the first audio according to the gain coefficient to obtain a corresponding first gain audio and sends it to the signal processor; S440. The pickup controller performs delay alignment on the second audio according to a preset delay duration to obtain a corresponding auxiliary audio; S450: The signal processor performs blocking filtering on the first audio and the independent pickup audio acquired by the independent omnidirectional microphone according to a preset blocking matrix, to obtain the corresponding blocking audio with the original audio in the target direction removed; S460: The signal processor performs adaptive filtering enhancement on the first gain audio according to a preset filtering enhancement strategy and the blocking audio, to obtain the corresponding first filtered audio and filter coefficients, and sends them to the pickup controller; S470: The pickup controller performs mixing processing on the first filtered audio and the auxiliary audio according to a preset ratio value, to obtain the corresponding mixed audio; S480: The pickup controller performs post-filtering processing on the mixed audio according to the filter coefficients, to obtain the corresponding target audio.

[0079] In addition, the unidirectional microphone in Figure 12 can also be replaced by a composite differential pickup array. In this case, apart from adding audio composite processing to the first and second sub-audio frequencies obtained by the two unidirectional microphones of the composite differential pickup array, the other processing steps are completely the same as those in Figure 13.

[0080] The steplessly adjustable pickup direction array pickup method disclosed in the above embodiments includes: receiving a first audio signal from a unidirectional microphone and a second audio signal from an omnidirectional microphone and calculating a gain coefficient; performing gain processing on the first audio signal according to the gain coefficient to obtain a first gain audio signal; performing delay alignment on the second audio signal to obtain an auxiliary audio signal; performing mixing processing on the first gain audio signal and the auxiliary audio signal according to a ratio value to obtain a mixed audio signal; and performing noise reduction processing on the mixed audio signal according to a noise reduction strategy and the gain coefficient to obtain the corresponding target audio signal. This method, combined with an array pickup device that can steplessly adjust the pickup direction, utilizes two or more microphones with different pickup directions to form a pickup array and employs a specific microphone array algorithm to achieve a wide range of stepless adjustment of the pickup directionality. Furthermore, combined with an array post-processing noise reduction algorithm, it achieves stronger noise reduction capabilities than traditional single-microphone or simple array pickup.

[0081] This application also provides an array microphone with infinitely adjustable pickup direction. The array microphone includes a device body 1, a pickup controller 10, and a directional adjustable pickup array 20. The pickup controller 10 in the infinitely adjustable pickup array microphone is used to execute the method steps in the aforementioned infinitely adjustable pickup array method that correspond to the directional adjustable pickup array 20 and can be executed within the pickup controller 10. Specifically, please refer to FIG15, which is a schematic block diagram of the array microphone with infinitely adjustable pickup direction provided in this application embodiment.

[0082] As shown in Figure 15, the pickup controller 10 is disposed inside the device body 1; the directional adjustable pickup array 20 consists of a unidirectional microphone 21 and an omnidirectional microphone 22; the unidirectional microphone 21 and the omnidirectional microphone 22 are both disposed on the same outer surface of the device body 1; the pickup direction of the unidirectional microphone 21 is opposite to that of the omnidirectional microphone 22; the pickup controller 10 is communicatively connected to the unidirectional microphone 21 and the omnidirectional microphone 22 respectively.

[0083] The infinitely adjustable pickup array can be a small microphone, a microphone pen, a wireless lavalier microphone, a microphone headphone box, a microphone watch, or other similar devices.

[0084] By adding an independent pickup device 40 to the array pickup device in Figure 15, the array pickup device structure shown in Figure 17 can be obtained. The independent pickup device 40 includes a signal processor 41 and an independent omnidirectional microphone 42, which are connected for communication. The signal processor 41 is connected to the pickup controller 42 and the pickup controller 10. The device body 1 is provided with a fixing component to secure the independent pickup device 40. In use, the independent pickup device 40 is detached from the device body 1 for independent pickup and combined with the device body 1. The signal processor 41 in the independent pickup device 40 can then be used to execute the method steps corresponding to the directional adjustable pickup array 20 in the aforementioned array pickup method with stepless pickup direction adjustment, and which can be executed within the signal processor 41. The independent pickup device 40 can be set as an independent small microphone. The independent pickup device 40 can be detachably mounted on the device body 1. When needed, the independent pickup device 40 can be detached from the device body 1 and used for independent pickup.

[0085] This application also provides an array pickup device with stepless adjustable pickup direction. The array pickup device includes a device body 1, a pickup controller 10, and a composite differential pickup array 30. The pickup controller 10 in the array pickup device with stepless adjustable pickup direction is used to execute the method steps in the aforementioned array pickup method with stepless adjustable pickup direction that correspond to the composite differential pickup array 30 and can be executed within the pickup controller 10. Specifically, please refer to FIG16, which is a schematic block diagram of the array pickup device with stepless adjustable pickup direction provided in this application embodiment.

[0086] As shown in Figure 16, the pickup controller 10 is disposed inside the device body 1; the composite differential pickup array 30 consists of a composite pickup microphone 31 and an omnidirectional microphone 22, wherein the composite pickup microphone 31 is composed of two omnidirectional microphones symmetrically combined; the composite pickup microphone 31 and the omnidirectional microphone 22 are both disposed on the same outer surface of the device body 1; the three omnidirectional microphones 22 are arranged in a straight line, and the distance between the two omnidirectional microphones in the composite pickup microphone 31 is smaller than the distance between the composite pickup microphone 31 and the individually disposed omnidirectional microphone 22; the pickup controller 10 is communicatively connected to the three omnidirectional microphones 22 respectively.

[0087] By adding an independent pickup device 40 to the array pickup device in Figure 16, the array pickup device structure shown in Figure 18 can be obtained. The independent pickup device 40 includes a signal processor 41 and an independent omnidirectional microphone 42, which are connected for communication. The signal processor 41 is connected to the pickup controller 42 and the pickup controller 10. The device body 1 is provided with a fixing component for fixing the independent pickup device 40. In use, the independent pickup device 40 is detached from the device body 1 for independent pickup and combined with the device body 1. The signal processor 41 in the independent pickup device 40 can then be used to execute the method steps corresponding to the composite differential pickup array 30 and executed within the signal processor 41 in the aforementioned array pickup method with steplessly adjustable pickup direction.

[0088] The steplessly adjustable pickup array microphone device provided in this application embodiment applies the aforementioned steplessly adjustable pickup array microphone method. It receives a first audio signal from a unidirectional microphone and a second audio signal from an omnidirectional microphone, calculates a gain coefficient, performs gain processing on the first audio signal based on the gain coefficient to obtain a first gain audio signal, performs delay alignment on the second audio signal to obtain an auxiliary audio signal, performs mixing processing on the first gain audio signal and the auxiliary audio signal based on a proportional value to obtain a mixed audio signal, and performs noise reduction processing on the mixed audio signal based on a noise reduction strategy and the gain coefficient to obtain the corresponding target audio signal. This method, combined with a steplessly adjustable pickup array microphone device, utilizes two or more microphones with different pickup directions to form a pickup array and employs a specific microphone array algorithm to achieve a wide range of stepless adjustment of pickup directionality. Simultaneously, combined with an array post-processing noise reduction algorithm, it achieves stronger noise reduction capabilities than traditional single-microphone or simple array microphones.

[0089] The above-described array pickup method with stepless adjustable pickup direction can be implemented as a computer program, which can run on the computer device shown in Figure 19.

[0090] Please refer to Figure 19, which is a schematic block diagram of a computer device provided in an embodiment of this application. This computer device may be an MCU chip used to execute a continuously adjustable array pickup method to achieve continuously adjustable array pickup processing.

[0091] Referring to Figure 19, the computer device 500 includes a processor 502, a memory, and a communication interface 505 connected via a communication bus 501. The memory may include a storage medium 503 and internal memory 504.

[0092] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute an array pickup method with infinitely adjustable pickup direction. The storage medium 503 may be a volatile storage medium or a non-volatile storage medium.

[0093] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0094] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an array pickup method with infinitely adjustable pickup direction.

[0095] The communication interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art will understand that the structure shown in FIG19 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0096] The processor 502 is used to run the computer program 5032 stored in the memory to implement the corresponding functions in the above-mentioned array pickup method with stepless adjustable pickup direction.

[0097] Those skilled in the art will understand that the embodiments of the computer device shown in FIG19 do not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only a memory and a processor. In such embodiments, the structure and function of the memory and processor are consistent with those shown in FIG19, and will not be repeated here.

[0098] It should be understood that, in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0099] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium may be volatile or non-volatile. The computer-readable storage medium stores a computer program, wherein when executed by a processor, the computer program implements the steps included in the described steplessly adjustable array pickup method.

[0100] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, or it may be an electrical, mechanical, or other form of connection.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a computer-readable storage medium and includes several instructions to cause a computer device (which may be a circuit board, data processing chip, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0105] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for array pickup with infinitely adjustable pickup directionality, characterized in that, The method is applied to a pickup controller connected with a directional adjustable pickup array, the directional adjustable pickup array is composed of a single directional microphone and a non-directional microphone, and the method comprises: If the first audio from the single directional microphone and the second audio from the non-directional microphone are received, a gain coefficient corresponding to the first audio and the second audio is calculated according to a preset audio gain strategy; The first audio is gain-processed according to the gain coefficient to obtain corresponding first gain audio; The second audio is time-delayed and aligned according to a preset delay time to obtain corresponding auxiliary audio; The first gain audio and the auxiliary audio are mixed according to a preset proportion value to obtain corresponding mixed audio; The mixed audio is noise-reduced according to a preset noise reduction strategy and the gain coefficient to obtain corresponding target audio.

2. An array pickup method in which the pickup direction is infinitely adjustable, characterized by comprising: The method is applied to a pickup controller connected with a composite differential pickup array, the composite differential pickup array is composed of a composite pickup microphone and a non-directional microphone, the composite pickup microphone is composed of two symmetrical non-directional microphones, and the method comprises: If the first sub-audio and the second sub-audio from the composite pickup microphone and the second audio from the non-directional microphone are received, a first audio is obtained by audio compounding the first sub-audio and the second sub-audio according to a preset directional adjustment parameter; A gain coefficient corresponding to the first audio and the second audio is calculated according to a preset audio gain policy; The first audio is gain-processed according to the gain coefficient to obtain corresponding first gain audio; The second audio is time-delayed and aligned according to a preset delay time to obtain corresponding auxiliary sound; The first gain audio and the auxiliary audio are mixed according to a preset proportion value to obtain corresponding mix audio; The mixed audio is noise-reduced according to a preset noise reduction strategy and the gain coefficient to get corresponding target audio.

3. The array pickup method of claim 1 or 2, wherein, The method further comprises: Obtaining first test audio and second test audio obtained by the microphone from the sound source at the predetermined position; the predetermined position is the position with the normal direction of the microphone and the distance from the microphone is the preset distance; the microphone line is the line between the single directional microphone and the non-directional microphone or the line between the single directional microphone and the composite pickup microphone; The calibration coefficient corresponding to the single directional microphone and the other microphone is allocated according to the volume of the first test audio and the second test audio, so that the average output volume of the first test audio and the second test audio after calibration is equal.

4. The array pickup method of claim 3, wherein, The gain coefficient corresponding to the first audio and the second audio is calculated according to the preset audio gain strategy, which comprises: The first energy function and the second energy function corresponding to the first audio and the second audio are calculated according to the average amplitude function in the audio gain strategy, respectively; According to a gain calculation formula in the audio gain strategy, the first energy function and the second energy function are calculated to obtain a gain coefficient corresponding to a current audio frame.

5. The array pickup method of claim 3, wherein, The gain coefficient corresponding to the first audio and the second audio is calculated according to a preset audio gain strategy, including: According to a transform calculation formula in the audio gain strategy, the first audio and the second audio are respectively subjected to short-time Fourier transform to obtain corresponding first frequency domain signals and second frequency domain signals; According to an average amplitude function in the audio gain strategy, a first energy function and a second energy function corresponding to the first frequency domain signals and the second frequency domain signals are respectively calculated; According to a gain calculation formula in the audio gain strategy, the first energy function and the secondenergy function are calculated to obtain a frequency domain sub-band gain coefficient as a gain coefficient corresponding to a current audio frame.

6. An array pickup method in which the pickup direction is infinitely adjustable, characterized by comprising: The method is applied to a pickup controller and a signal processor, the pickup controller is in communication connection with a directional adjustable pickup array and the signal processor respectively, the directional adjustable pickup array is composed of a single directional microphone and a non-directional microphone, the signal processor is in communication connection with an independent non-directional microphone to obtain corresponding independent pickup audio from the independent non-directional microphone, and the method comprises: If the pickup controller receives first audio from the single directional microphone and second audio from the non-directional microphone, the first audio is sent to the signal processor; The pickup controller calculates a gain coefficient corresponding to the first audio and the second audio according to a preset audio gain strategy; The pickup controller performs gain processing on the first audio according to the gain coefficient to obtain corresponding first gain audio and sends the first gain audio to the signal processor; The pickup controller performs delay alignment on the second audio according to a preset delay time length to obtain corresponding auxiliary audio; The signal processor performs blocking filter processing on the first audio and the independent pickup audio obtained by the independent non-directional microphone according to a preset blocking matrix to obtain blocking audio in which a target direction original audio is removed; The signal processor performs adaptive filter enhancement on the first gain audio according to a preset filter enhancement strategy and the blocking audio to obtain corresponding first filter audio and filter coefficients and sends the first filter audio and the filter coefficients to the pickup controller; The pickup controller performs audio mixing processing on the first filter audio and the auxiliary audio according to a preset proportion value to obtain corresponding mixed audio; The pickup controller performs post-filter processing on the mixed audio according to the filter coefficients to obtain corresponding target audio.

7. A method for array pickup with infinitely adjustable pickup directionality, characterized in that, The method is applied to a pickup controller and a signal processor, the pickup controller is incommunication connection with a composite differential pickup array and the signal processor respectively, the composite differential pickup array is composed of a composite pickup microphone and a non-directional microphone, the composite pickup microphone is composed of two symmetrical non-directional microphones, and the signal processor is in communication connection with an independent non-directional microphone to obtain corresponding independent pickupaudio from the independent non-directional microphone, and the method comprises: If the pickup controller receives the first sub-audio and the second sub-audio from the compound pickup microphone and the second audio from the non-directional microphone, the first sub-audio and the second sub-audio are combined according to the preset directivity adjustment parameter to obtain the corresponding first audio and sent to the signal processor; The pickup controller calculates the gain coefficient corresponding to the first audio and the second audio according to the preset audio gain strategy; The pickup controller performs gain processing on the first audio according to the gain coefficient to obtain the corresponding first gain audio and sends it to the signal processor; The pickup controller performs delay alignment on the second audio according to the preset delay time to obtain the corresponding auxiliary audio; The signal processor performs blocking filter processing on the first audio and the independent pickup audio collected by the independent non-directional microphone according to the preset blocking matrix to obtain the blocking audio that removes the blocking audio of the target direction original audio; The signal processor performs adaptive filter enhancement on the first gain audio according to the preset filter enhancement strategy and the blocking audio to obtain the corresponding first filter audio and filter coefficient and sends it to the pickup controller; The pickup controller performs audio mixing processing on the first filter audio and the auxiliary audio according to the preset proportion value to obtain the corresponding mixed audio; The pickup controller performs post-filter processing on the mixed audio according to the filter coefficient to obtain the corresponding target audio.

8. A pickup array device with infinitely adjustable pickup directionality, characterized in that, The device includes a device body, a pickup controller, and a directivity adjustable pickup array, and the pickup controller is used to perform the array pickup method of stepless adjustable directivity as claimed in any one of claims 1, 3-5; The pickup controller is arranged in the device body; The directivity adjustable pickup array is composed of a single directional microphone and a non-directional microphone; the single directional microphone and the non-directional microphone are arranged on the same outer surface of the device body; the pickup direction of the single directional microphone is away from the non-directional microphone; The pickup controller is communicatively connected with the single directional microphone and the non-directional microphone respectively.

9. An array pickup device in which the pickup direction is infinitely adjustable, characterized by comprising: The device includes a device body, a pickup controller, and a compound differential pickup array, and the pickup controller is used to perform the array pickup method of stepless adjustable direction as claimed in any one of claims 2-5; The pickup controller is arranged in the device body; The compound differential pickup array is composed of a compound pickup microphone and a non-directional microphone, and the compound pickup microphone is composed of two symmetrical non-directional microphones; the compound pickup microphone and the non-directional microphone are arranged on the same outer surface of the device body; three non-directional microphones are arranged in a straight line, and the distance between the two non-directional microphones in the compound pickup microphone is smaller than the distance between the compound pickup microphone and the separately arranged non-directional microphone; The pickup controller is communicatively connected with the three non-directional microphones respectively.

10. The array pickup device of claim 8 or 9, wherein, The device further includes an independent pickup device, which includes a signal processor and an independent non-directional microphone communicatively connected, and the signal processor is communicatively connected with the pickup controller; The device body is provided with a fixing assembly for fixing the independent sound pickup device; in use, the independent sound pickup device is detached from the device body for independent sound pickup and combined use with the device body.

Images