Beamforming method, device and storage medium for microphone or speaker array

By setting the zero position and minimum norm constraints, combined with Jacobi-Anger series expansion, beamforming of microphone or speaker array is achieved, solving the problems of insufficient pick-up performance of single microphone and difficult to maintain beam width, and realizing the entire frequency band Constant beam width and strong anti-interference performance.

CN115474136BActive Publication Date: 2024-11-01COLSONIC SUZHOU ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211303244.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-24
Publication Date
2024-11-01
Estimated Expiration
2042-10-24

AI Technical Summary

Technical Problem

In complex acoustic environments, single microphone pickup performance is affected by background noise, interference, and reverb, resulting in reduced speech clarity, and existing differential beamforming methods are difficult to maintain a constant beam width at mid- and high frequencies.

Method used

By setting N zero positions, satisfying the M≥2N+1 condition, converting the target beam expression and performing ±2N order truncation, using the minimum norm constraint to obtain the weight vector, combined with Jacobi-Anger series expansion, realizing the microphone or speaker array Beamforming.

Benefits of technology

Maintain a constant beam width throughout the frequency band, improve low-frequency direction and anti-interference performance, expand the frequency usage range, and achieve broadband sound pickup or sound directional playback effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115474136B_ABST
    Figure CN115474136B_ABST
Patent Text Reader

Abstract

The present invention discloses a beamforming method for a microphone array or a speaker array. The beamforming method comprises: setting N zero-point positions, each of which deviates from the desired target direction of the microphone / speaker array, wherein the number of microphones in the microphone array or the number of speakers in the speaker array is M, and the condition M≥2N+1 is satisfied; converting the target beam expression according to the N zero-point positions; comparing the actual beam pattern expression of the microphone / speaker array with the target beam expression after ±2N-order truncation, and obtaining a weight vector through a minimum norm constraint; converting the sound signal picked up by the microphone array into the frequency domain, and converting it into the time domain after multiplying and superimposing with the corresponding weights; or, converting the playback sound source into the frequency domain, and converting it into the time domain after multiplying with the weight of the speaker array, to obtain the sound played by the speaker unit. The method of the present invention achieves a good constant beam width in the entire frequency band under the premise of ensuring that only the zero-point position needs to be set.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a beam forming method of a microphone array or a loudspeaker array, electronic equipment and a storage medium. Background Art

[0002] The beamforming method can be applied to the sound pickup of a microphone array or the playback of a speaker array.

[0003] For microphone arrays, in complex acoustic environments, factors such as background noise, interference, and reverberation will deteriorate the microphone's pickup performance, reduce the clarity and intelligibility of speech, and make single microphone pickup unable to meet daily needs. Compared with the pickup effect of a single microphone, microphone arrays combined with beamforming algorithms can enhance signals from the target direction, suppress signal gain from non-target directions, improve the signal-to-noise ratio of output speech, and improve speech quality by fusing space-time information. Therefore, they are widely used in speech enhancement and speech recognition. The differential beamforming design algorithm has a high directivity index and can achieve a broadband constant beam width, which can effectively reduce speech distortion. Therefore, it has received widespread attention among many fixed beamforming design algorithms.

[0004] For speaker arrays, in applications such as personal listening areas and square dance performances, the speakers are required to emit sound only in the target direction without interfering with other areas. Differential beamforming has also received widespread attention in speaker arrays because of its constant beam width.

[0005] In broadside array design, the differential beam design method based on zero-point constraints has been widely used in engineering practice because of its simple parameter setting (only the zero-point position needs to be set). However, it is more sensitive to errors at low frequencies. By increasing the number of units and adopting the minimum norm constraint, the low-frequency robustness can be effectively improved, but its constant beam width will be destroyed at medium and high frequencies. Summary of the invention

[0006] In view of the above problems, the present invention provides a beamforming method, electronic device and storage medium for a microphone array or a speaker array, which can achieve a good constant beam width in the entire frequency band while ensuring that only a zero point position needs to be set.

[0007] According to a first aspect of the present invention, a beamforming method for a linear microphone array or a speaker array comprises the following steps:

[0008] S1. Setting N zero point positions, each of which deviates from a desired target direction of a microphone array or a speaker array, wherein the number of microphones in the microphone array or the number of speakers in the speaker array is M, and a condition M≥2N+1 is satisfied;

[0009] S2, converting the target beam expression according to the N zero point positions;

[0010] S3, the actual beam pattern expression of the microphone array or the loudspeaker array is truncated to ±2N order and compared with the target beam pattern expression, and the weight vector w=[w1(k),…,w M-1 (k),w M (k)] T , w m (k) is the weight of the mth microphone or speaker channel, m = 1, ..., M, T is the transpose;

[0011] S4. Convert the sound signal picked up by the microphone array into the frequency domain, multiply the frequency domain sound signal of each microphone channel by the corresponding weight, superimpose and convert to the time domain to obtain the final output sound signal; or, convert the playback sound source into the frequency domain, multiply it by the weight corresponding to each channel of the speaker array and convert it to the time domain to obtain the sound played by each speaker unit.

[0012] Preferably, the weight vector w is calculated by the following formula:

[0013]

[0014] Among them, the matrix Φ is obtained according to the actual beam pattern expression, and the vector Obtained according to the target beam expression, H is the conjugate transpose.

[0015] More preferably, the actual beam pattern expression is expanded through Jacobi-Anger series to obtain the matrix Φ.

[0016] In a specific and preferred embodiment, the actual beam pattern expression is: Where j is an imaginary number, k = ω / c, ω is the angular frequency, c is the speed of sound, x m is the position of the microphone or speaker, w m (k) is the weight of the corresponding microphone or speaker;

[0017] The beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the Bessel function of the first kind of order n;

[0018] The beam expression of Jacobi-Anger series expansion is truncated to ±2N order to obtain the matrix Φ shown in the following formula:

[0019]

[0020] More preferably, the target beam expression is written as a series expression, and the target beam expression is expressed as jnθ The coefficients of the series expansion constitute the vector

[0021] In a specific and preferred embodiment, the target beam expression Where θ is the beam direction, ∏ is the multiplication of all elements, β n and zero position θ N,n The relationship is β n =cos(θ N,n ), n=1,2,…,N;

[0022] Will Substitute the target beam expression and expand it to write a series expression Where j is an imaginary number, n is an integer index subscript, press e jnθ Coefficients of series expansion r is an integer index subscript, is the number of combinations, ! is expressed as a factorial; α r is the intermediate coefficient, and its expression is:

[0023] Represents r different integer subscripts, and the value is not less than 1 and not greater than r;

[0024] The vector is obtained according to the following formula

[0025] Preferably, in step S1, the M microphones or loudspeakers are arranged in sequence at equal intervals along a straight line, and the desired target direction is a direction perpendicular to the straight line and passing through the center of the microphone array or loudspeaker array.

[0026] More preferably, the zero point position is an angle deviating from the straight line, and N zero point positions θ N,1 ,θ N,2 ,…θ N,N Satisfy 0≤θ N,1 <θ N,2 <…<θ N,N <90,n=1,2,…,N.

[0027] According to a second aspect of the present invention, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the beamforming method as described above when executing the program.

[0028] According to a preferred and specific aspect, the electronic device is a remote conference device.

[0029] According to a third aspect of the present invention, a computer-readable storage medium stores a computer program, which implements the beamforming method as described above when executed by a processor.

[0030] The present invention adopts the above solution, which has the following advantages compared with the prior art:

[0031] The beamforming method of the present invention only needs to set the zero point position, and at the same time, comparative design is performed in the mode domain through conversion, thereby retaining the advantages of simple parameter design setting and easy engineering implementation, and at the same time, a good constant beam width can be obtained in the entire frequency band, the frequency use range is expanded, and a good directivity effect is achieved at low frequencies, and the anti-interference performance is strong, so that a good broadband sound pickup effect or a sound directional playback effect can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0033] Figure 1 FIG. 4 is a schematic diagram of a linear microphone array according to an embodiment of the present invention.

[0034] Figure 2a is the beam pattern of the traditional zero-point constraint method; Figure 2b is the beam pattern of the traditional minimum norm method; Figure 2c is a beam diagram of a method according to an embodiment of the present invention.

[0035] Figure 3 It is a white noise gain comparison curve diagram of the traditional zero-point constraint method, the traditional minimum norm method and the method of the embodiment of the present invention.

[0036] Figure 4 It is a directivity index comparison curve diagram of the traditional zero-point constraint method, the traditional minimum norm method and the method of the embodiment of the present invention. DETAILED DESCRIPTION

[0037] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] This embodiment provides a beamforming method. This embodiment takes a linear microphone array as an example to describe the beamforming method in detail. Figure 1 As shown, the linear microphone array includes a plurality of microphones 101, specifically M, that is, the number of microphones 101 is an odd number. The plurality of microphones 101 are arranged in sequence along a straight line 100, preferably at equal intervals. The desired target direction 200 is a direction perpendicular to the straight line 100, and the desired target direction 200 passes through the center of the microphone array.

[0039] The beamforming method obtains the sound signal finally output by the microphone array by multiplying the sound signal picked up by the microphone array after conversion to the frequency domain with the weight of the corresponding microphone channel, superimposing it and then converting it to the time domain. The vector of the weight of each microphone channel is obtained specifically by the following method. In some other embodiments, the beamforming algorithm is applied to the sound projection of the speaker array, specifically by converting the playback sound source into the frequency domain, and multiplying it with the weight corresponding to each channel of the speaker array and converting it into the time domain to obtain the sound played by each speaker unit. The vector of the weight of each speaker channel can also be obtained by the following method.

[0040] (1) Set N zero positions

[0041] According to actual needs, determine the N zero point positions θ N,1 ,θ N,2 ,…θ N,N , n is an integer index subscript, specifically n=1,2,…,N. The zero point position must satisfy 0≤θ N,1 <θ N,2 <…<θ N,N <90. The number of microphones and the number of zero positions must satisfy the condition M≥2N+1. Figure 1 As shown, each zero point position 300 deviates from the desired target direction 200 of the microphone array, specifically refers to an angle deviating from the straight line 100 .

[0042] (2) Convert the target beam expression according to the N zero point positions

[0043] Convert to target beam expression based on zero point position Where θ is the beam direction, ∏ is the multiplication of all elements, β n and zero position θ N,n The relationship is β n =cos(θ N,n ), n=1,2,…,N.

[0044] Will Substitute the target beam expression and expand it to write a series expression Where j is an imaginary number, n is an integer index subscript, press ejnθ Coefficients of series expansion for integer index subscripts, is the number of combinations, ! is expressed as a factorial; α r is the intermediate coefficient, and its expression is:

[0045] Represents r different integer subscripts, and the value is not less than 1 and not greater than r;

[0046] The vector is obtained according to the following formula

[0047] (3) The actual beam pattern expression of the microphone array is truncated by ±2N orders

[0048] The actual beam pattern expression is Where j is an imaginary number, k = ω / c, ω is the angular frequency, c is the speed of sound, x m is the microphone position, w m (k) is the weight of the corresponding microphone;

[0049] The beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the Bessel function of the first kind of order n;

[0050] The beam expression of Jacobi-Anger series expansion is truncated to ±2N order to obtain the matrix Φ shown in the following formula:

[0051]

[0052] (4) is compared with the target beam expression and the weight vector w = [w1(k),…,w M-1 (k),w M (k)] T , T is the transpose.

[0053] The weight vector w is calculated as follows:

[0054]

[0055] Among them, the matrix Φ is obtained through the above step (3) according to the actual beam pattern expression, and the vector Obtained through the above step (2) according to the target beam expression, H is the conjugate transpose.

[0056] Simulation Example

[0057] The microphone array used in the simulation is a seven-element linear array with a spacing of 5 cm. The frequency range considered is [100,4000] Hz, covering the language sound frequency band. The desired direction is 90 degrees, and the zero position is set to 10 degrees and 30 degrees.

[0058] Three methods are used for beam design: beam design based on the traditional zero-point constraint method, beam design based on the traditional minimum norm method, and the beam design method proposed in this embodiment. The first two design methods can be found in the document "Chen, Jingdong. Benesty, Jacob. Pan, Chao. On the design and implementation of linear differential microphone arrays. J. Acoust. Soc. Am., Vol. 136, No. 6, Dec. 2014."

[0059] The evaluation indicators are beam pattern, white noise gain (WNG) and directivity index (DI); WNG is used to describe the system's ability to resist system errors such as microphone self-noise and mismatch. The larger its value, the stronger the error resistance performance; DI is used to indicate directivity. The larger its value, the better the directivity.

[0060] Figure 2a to Figure 2c The following are the beam diagrams of the above three methods at different frequencies. The color depth in the figure represents the size of the beam response, and the brighter the value, the larger the value. Figure 2a to Figure 2c , Figure 3 and Figure 4 It can be seen that the design method based on zero-point constraint basically achieves constant beamwidth in the entire frequency band, but its biggest drawback is that the WNG value is low at low frequencies and the robustness is poor. The minimum norm method uses redundant speakers to improve the WNG value at low frequencies and improve the error resistance performance. At the same time, after 2000Hz, the DI value gradually increases, which indicates that the directivity is enhanced, but it will destroy the constant beamwidth, which is inconsistent with the design concept. Figure 2b It can also be seen from the beam diagram that the main lobe becomes narrower at high frequencies. Using the design method of this embodiment, the anti-error performance can be effectively improved at low frequencies (the WNG value is basically the same as the WNG value of the minimum norm method, and is much larger than the WNG value of the zero-point constraint method), while the beam width can be kept basically unchanged at high frequencies, integrating the advantages of the first two algorithms and avoiding the defects of the aforementioned algorithms. At the same time, the parameter setting is simple and easy to implement in engineering.

[0061] The beamforming method of the microphone array of this embodiment has better directivity effect at low frequencies and stronger anti-interference performance, and at the same time, the beam width is basically constant throughout the entire frequency band.

[0062] The above embodiment is only for illustrating the technical concept and features of the present invention, and is a preferred embodiment. Its purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and it cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A beamforming method for a linear microphone array or a loudspeaker array, It is characterized in that The steps include: S1. Setting N zero point positions, each of which deviates from a desired target direction of a microphone array or a speaker array, wherein the number of microphones in the microphone array or the number of speakers in the speaker array is M, and a condition M≥2N+1 is satisfied; S2, converting the target beam expression according to the N zero point positions; S3, the actual beam pattern expression of the microphone array or the loudspeaker array is truncated to ±2N order and then compared with the target beam pattern expression, and a weight vector w=[w 1 (k),…,w M-1 (k),w M (k)] T , w m (k) is the weight of the mth microphone or speaker channel, m = 1, ..., M, T is the transpose; S4, converting the sound signal picked up by the microphone array into the frequency domain, multiplying the frequency domain sound signal of each microphone channel by the corresponding weight, superimposing and converting to the time domain, to obtain the final output sound signal; or, converting the playback sound source into the frequency domain, and multiplying it with the weight corresponding to each channel of the speaker array and then converting it to the time domain, to obtain the sound played by each speaker unit; The weight vector w is calculated as follows: Among them, the matrix Φ is obtained according to the actual beam pattern expression, and the vector Obtained according to the target beam expression, H is the conjugate transpose; The actual beam pattern expression is expanded by Jacobi-Anger series to obtain the matrix Φ; The actual beam pattern expression is Where j is an imaginary number, k = ω / c, ω is the angular frequency, c is the speed of sound, x m is the position of the microphone or speaker, w m (k) is the weight of the corresponding microphone or speaker; The beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the Bessel function of the first kind of order n; The beam expression of Jacobi-Anger series expansion is truncated to ±2N order to obtain the matrix Φ shown in the following formula: Write the target beam expression as a series expression and convert the target beam expression into jnθ The coefficients of the series expansion constitute the vector The target beam expression Where θ is the beam direction, ∏ is the multiplication of all elements, β n and zero position θ N,n The relationship is β n =cos(θ N,n ), n=1,2,…,N; Will Substitute the target beam expression and expand it to write a series expression Where j is an imaginary number, n is an integer index subscript, press e jnθ Coefficients of series expansion r is an integer index subscript, is the number of combinations, ! is expressed as a factorial; α r is the intermediate coefficient, and its expression is: i 1 ,i 2 …,i r Represents r different integer subscripts, and the value is not less than 1 and not greater than r; The vector is obtained according to the following formula 2. The beamforming method according to claim 1, It is characterized in that In step S1, the M microphones or loudspeakers are arranged in sequence along a straight line at equal intervals, and the desired target direction is a direction perpendicular to the straight line and passing through the center of the microphone array or loudspeaker array.

3. The beamforming method according to claim 2, It is characterized in that The zero point position is the angle of deviation from the straight line, N zero point positions θ N,1 ,θ N,2 ,…θ N,N Satisfy 0≤θ N,1 <θ N,2 <…<θ N,N <90.

4. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the beamforming method according to any one of claims 1 to 3 is implemented.

5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the beamforming method according to any one of claims 1 to 3.