Constant beam shape design method based on differential array, audio equipment and storage medium
Through the constant beam shape design method based on differential arrays, the optimal problem solution and Jacobi-Anger series expansion are used to sacrifice a small amount of white noise gain, and the problem of unstable beam shape in the frequency band in the prior art is solved, and the medium and high frequency is realized The significant improvement at the location and the constant beam shape improve the performance of the microphone and speaker arrays.
Patent Information
- Application Number
- CN202411298779.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2024-12-31
AI Technical Summary
The prior art is difficult to maintain a constant beam shape throughout the frequency band, especially at mid-to-high frequencies, affecting the sound pickup and playback performance of the microphone array and speaker array.
Using a constant beam shape design method based on differential arrays, the weight vector of the microphone or speaker array is obtained through the optimal problem solution, sacrificing a small amount of white noise gain performance, optimizing the beam shape on the entire frequency band, using Jacobi-Anger series expansion and matrix Φ Processing to ensure the constant beam shape.
The beam shape is basically kept unchanged throughout the frequency band, especially at mid-to-high frequencies, improving the voice quality and error resistance of the microphone array and speaker array.
Smart Images
Figure CN119233159A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of microphone array sound pickup control or loudspeaker array playback control, and relates to a constant beam shape design method based on a differential array, an audio device and a storage medium. Background Art
[0002] The constant beam shape design method can be applied to the pickup of microphone arrays or the playback of speaker arrays. For microphone arrays, in complex acoustic environments, factors such as background noise, interference, and reverberation will deteriorate the microphone's pickup performance, reduce the clarity and intelligibility of speech, and cause a single microphone to be unable to meet daily needs. Compared with the pickup effect of a single microphone, the microphone array combined with the beamforming algorithm can enhance the signal from the target direction by fusing space-time information, suppress the gain of the signal from the non-target direction, improve the signal-to-noise ratio of the output speech, and improve the speech quality. Therefore, it is widely used in speech enhancement and speech recognition. The differential beamforming design algorithm has a high directivity index and can achieve a broadband constant beam width, which can effectively reduce speech distortion. Therefore, it has received widespread attention among many fixed beamforming design algorithms.
[0003] For speaker arrays, in applications such as personal listening areas and square dance performances, the speakers are required to emit sound only in the target direction without interfering with other areas. Differential beamforming has also received widespread attention in speaker arrays because of its constant beam width.
[0004] In practice, in order to improve the anti-error performance, the number of microphone units is usually increased and the minimum norm constraint method is used to improve the robustness performance, but the constant beam width at the mid-high frequency will be destroyed. Based on this, the applicant disclosed a beamforming method for a microphone or speaker array in patent CN115474136A, which achieves a good constant beam width in the entire frequency band while ensuring that only the zero point position needs to be set. However, in practical applications, the inventors found that there is still room for improvement in the constant beam shape in the entire frequency band.
[0005] The above information disclosed in the Background section is only for enhancement of understanding of the background of the present application and therefore it may contain information that does not constitute the prior art that is already known to a person of ordinary skill in the art. Summary of the invention
[0006] In view of the above problems, the present invention provides a constant beam shape design method based on a differential array, an audio device and a storage medium, which further improve the constant beam shape over the entire frequency band by sacrificing a small amount of white noise gain performance.
[0007] The first aspect of the present invention adopts the following technical solution:
[0008] A constant beam shape design method based on a differential array includes solving the following optimal problem to obtain a weight vector of a microphone array or a speaker array:
[0009]
[0010] in,
[0011]
[0012] w is the weight vector, H is the conjugate transpose, θ is the beam direction, is the target beam expression; j is an imaginary number, x M is the position of the microphone or speaker, k = ωc, ω is the angular frequency, c is the speed of sound, and T is the transpose;
[0013] st represents the constraint condition, and the expression of the matrix Φ is:
[0014]
[0015] N is an integer index, J N (·) is the Bessel function of the first kind of order N;
[0016] is the coefficient vector of the target beam;
[0017] ε opt is the optimal white noise gain, and loss is the allowable loss of white noise gain performance.
[0018] In a preferred embodiment, the optimal white noise gain expression is defined as: in,
[0019] In a preferred embodiment, the coefficient vector of the target beam is
[0020] In a more preferred embodiment, the target beam expression is:
[0021]
[0022] Where n = -N,…,N.
[0023] In a preferred embodiment, the microphone array includes 2M+1 microphones, or the speaker array includes 2M+1 speakers, and the condition M≥N+1 is satisfied.
[0024] In a preferred embodiment, the microphone array is a linear microphone array, or the speaker array is a linear speaker array. Further, 2M+1 microphones or speakers are arranged in sequence and at equal intervals along a straight line, and the desired target direction is a direction perpendicular to the straight line and passing through the center of the microphone array or speaker array. The zero point position deviates from the desired target direction of the microphone array or speaker array, specifically, an angle θ from the straight line.
[0025] In a preferred embodiment, the actual beam pattern expression is:
[0026]
[0027] where w m (k) is the weight of the corresponding microphone or speaker;
[0028] The beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the Bessel function of the first kind of order n, n = -N,…,N;
[0029] The matrix Φ is obtained by truncating the beam expression of the Jacobi-Anger series expansion by ±N orders.
[0030] In a preferred embodiment, the sound signal picked up by the microphone array is converted into the frequency domain, and the frequency domain sound signal of each microphone channel is multiplied by the corresponding weight, superimposed and converted into the time domain to obtain the final output sound signal; or, the playback sound source is converted into the frequency domain, and multiplied with the weights corresponding to each channel of the speaker array and converted into the time domain to obtain the sound played by each speaker unit.
[0031] According to a second aspect of the present invention, an audio device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the constant beam shape design method as described above when executing the program.
[0032] In a preferred embodiment, the audio device includes a microphone array, and the processor obtains a weight vector of the microphone array and processes the picked-up sound signal accordingly.
[0033] In a preferred and specific aspect, the audio device is a teleconferencing device.
[0034] In a preferred embodiment, the audio device includes a speaker array, and the processor obtains a weight vector of the speaker array and processes the audio signal accordingly, and then drives the speaker array to reproduce the sound.
[0035] In a preferred embodiment, the audio device further includes a multi-channel ADC digital-to-analog conversion module electrically connected to the microphone array, and the processor is electrically connected to the multi-channel ADC digital-to-analog conversion module.
[0036] In a preferred embodiment, the processor includes a DSP chip or a FPGA chip.
[0037] According to a third aspect of the present invention, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the constant beam shape design method as described above.
[0038] The present invention adopts the above solution, which has the following advantages compared with the prior art:
[0039] The constant beam shape design method of the present invention is based on a differential array and further optimizes and improves the constant beam shape performance by sacrificing a small amount of white noise gain, so that the beam shape can remain basically unchanged over the entire frequency band. Compared with existing methods, there is a significant improvement in the mid- and high-frequency ranges. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0041] Figure 1 FIG. 4 is a schematic diagram of a linear microphone array according to an embodiment of the present invention.
[0042] Figure 2 It is the beam diagram of the MM method of the comparative example.
[0043] Figure 3 2 is a beam diagram of an embodiment method.
[0044] Figure 4 It is the MSE curve of the MM method of comparative example and the method of embodiment.
[0045] Figure 5 The figure is a structural block diagram of an audio device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0046] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0047] The minimum norm beam design method based on zero-point constraints cannot maintain a constant beam shape over the entire frequency band, while the minimum norm differential beam design method based on modal constraints can improve related phenomena. To further improve the constant beam shape over the entire frequency band, this embodiment proposes an improved constant beam shape design method based on a differential array, which further improves the constant beam shape over the entire frequency band by sacrificing a small amount of white noise gain performance.
[0048] The following describes the constant beam shape design method in detail by taking a linear microphone array as an example. Figure 1 As shown, the linear microphone array includes a plurality of microphones 101, specifically 2M+1, that is, the number of microphones 101 is an odd number. The plurality of microphones 101 are sequentially arranged at intervals along the straight line 100, preferably arranged at equal intervals. The desired target direction 200 is a direction perpendicular to the straight line 100, and the desired target direction 200 passes through the center of the microphone array.
[0049] In the constant beam shape design method, the sound signal picked up by the microphone array after conversion to the frequency domain is multiplied by the weight of the corresponding microphone channel, superimposed and then converted to the time domain to obtain the sound signal finally output by the microphone array. The vector of the weight of each microphone channel is specifically obtained by the following method. In some other embodiments, the constant beam shape design method is applied to the sound projection of the speaker array, specifically by converting the playback sound source into the frequency domain, and multiplying it with the weight corresponding to each channel of the speaker array and converting it into the time domain to obtain the sound played by each speaker unit. The vector of the weight of each speaker channel can also be obtained by the following method.
[0050] (1)Reference Figure 1 As shown, θ is the wave arrival direction, and the end-fire direction of the linear microphone array is 0 degrees.
[0051] (2) The target beam expression is:
[0052]
[0053] Where n = -N, ..., N, is an integer index, N is the differential order, j is an imaginary number, and the differential order N and the coefficient vector of the target beam are determined according to actual needs. At the same time, the number of microphones is set to an integer 2M+1, and the condition M≥N+1 must be satisfied.
[0054] (3) The actual beam pattern expression is Where k = ωc, ω is the angular frequency, c is the speed of sound, x m is the microphone position, w m (k) is the weight of the corresponding microphone, m is an integer subscript, and the beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the nth-order Bessel function of the first kind.
[0055] (4) The minimum norm difference beam design method based on modal constraints is to compare the actual beam pattern Β(k,θ) with the target beam after ±N-order truncation, and minimize the weight norm. The optimal problem is:
[0056]
[0057] Where H is the conjugate transpose, and the weight vector w is expressed as:
[0058] w=[w -M (k),w -(M-1) (k),…,w0(k),…,w M-1 (k),w M (k)] T , T is the transpose, and the expression of the matrix Φ is:
[0059]
[0060] The solution to the above optimization problem is The optimal white noise gain expression is defined as
[0061]
[0062] (5) The above method can maximize the white noise gain, that is, the robustness is optimal, but there are still defects in the beam shape. Considering that a small amount of white noise gain can be sacrificed in practice, the performance of the constant beam shape can be further optimized and improved. The improved optimal problem is:
[0063]
[0064] in
[0065] dθ represents the integral of θ, and loss is the allowable loss of white noise gain performance in decibels.
[0066] The solution may be obtained by using a known method, for example, by using a convex optimization tool (https: / / cvxr.com / cvx / ).
[0067] simulation
[0068] The microphone array used in the simulation is an 11-element linear array with a spacing of 4 cm. The frequency range considered is [300, 4000] Hz, covering the language sound frequency band. The expected direction is 90 degrees, the beam width is 60 degrees, the order N is set to 2, and the coefficient vector design of the target beam can be found in the relevant paper (Eigen beam-space transformation based steerable differential beamforming for linear arrays). Here, the correlation coefficient is set to [0.2569, 0, -0.2260, 0, 0.1456]. Two methods are used for beam design, one is the method disclosed in patent CN115474136A (referred to as MM method), and the other is the method of the above embodiment, in which the parameter loss is set to 1.
[0069] Figure 2 and Figure 3 These are the beam diagrams of the above two methods at different frequencies. The color depth in the figure represents the size of the beam response, and the brighter the color, the larger the value. Figure 4 The MSE (minimum mean square error) curves of the two methods are shown. The upper curve represents the MSE curve of the MM method, and the lower curve represents the MSE curve of the method of this embodiment. Figures 2 to 4 It can be seen from the figure that the beam shape of the method of this embodiment remains basically unchanged in the entire frequency band, and compared with the MM method, there is a significant improvement in the mid-high frequency.
[0070] The method of the embodiment has a substantially constant beam shape in the entire frequency band by sacrificing a small amount of white noise gain performance.
[0071] This embodiment also provides an audio device to which the above-mentioned constant beam shape design method can be applied. Figure 5 As shown, the audio device includes a memory 4, a processor 3 and a computer program stored in the memory 4 and executable on the processor, and the processor 3 implements the constant beam shape design method as described above when executing the program. The audio device also includes a microphone array 1, and the processor 3 obtains a weight vector of the microphone array 1 and processes the picked-up sound signal accordingly. Specifically, the audio device is a teleconference device. The device has the advantages of being small in size and easy to install.
[0072] The audio device also includes a multi-channel ADC digital-to-analog conversion module 2 electrically connected to the microphone array 1, and the processor 3 is electrically connected to the multi-channel ADC digital-to-analog conversion module 2. The sound signals picked up by each microphone of the microphone array 1 are converted into digital signals by the multi-channel ADC digital-to-analog conversion module 2 and then sent to the processor 3. The processor 3 processes the multi-channel digital signals according to the calculated weight vector to obtain the final sound signal. The processor 3 specifically includes a DSP chip or an FPGA chip. The multi-channel ADC digital-to-analog conversion module 2 is specifically a multi-channel ADC digital-to-analog conversion chip.
[0073] In a preferred embodiment, the audio device includes a speaker array, and the processor obtains a weight vector pair of the speaker array, processes the audio signal accordingly, and then drives the speaker array to reproduce the sound.
[0074] This embodiment also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the constant beam shape design method described above is implemented. The computer-readable storage medium includes but is not limited to a read-only memory, a magnetic disk, or an optical disk.
[0075] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art. If a definition used herein conflicts or is inconsistent with a definition in other public documents, the definition used herein shall prevail.
[0076] As shown in this specification and claims, the terms "comprises" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements. The term "and / or" as used herein includes any combination of one or more of the relevant listed items.
[0077] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0078] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0079] In addition, each functional unit in the embodiment may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0080] The above embodiment is only for illustrating the technical concept and features of the present invention, and is a preferred embodiment. Its purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and it cannot be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be included in the protection scope of the present invention.
Claims
1. A constant beam shape design method based on differential array, characterized in that: This includes solving the following optimal problem to obtain the weight vector of the microphone array or the speaker array: in, w is the weight vector, H is the conjugate transpose, θ is the beam direction, is the target beam expression; j is an imaginary number, x M is the position of the microphone or speaker, k = ωc, ω is the angular frequency, c is the speed of sound, and T is the transpose; st represents the constraint condition, and the expression of the matrix Φ is: N is an integer index, J N (·) is the Bessel function of the first kind of order N; is the coefficient vector of the target beam; ε opt is the optimal white noise gain, and loss is the allowable loss of white noise gain performance.
2. The constant beam shape design method according to claim 1, characterized in that: The optimal white noise gain expression is defined as in, 3. The constant beam shape design method according to claim 1 or 2, characterized in that: Coefficient vector of the target beam 4. The constant beam shape design method according to claim 3, characterized in that: The target beam expression is: Where n = -N,…,N.
5. The constant beam shape design method according to claim 1, characterized in that: The microphone array includes 2M+1 microphones, or the speaker array includes 2M+1 speakers, and the condition M≥N+1 is satisfied.
6. The constant beam shape design method according to claim 1 or 5, characterized in that: The microphone array is a linear microphone array, or the speaker array is a linear speaker array.
7. The constant beam shape design method according to claim 1, characterized in that: The actual beam pattern expression is: where w m (k) is the weight of the corresponding microphone or speaker; The beam expression can be obtained by Jacobi-Anger series expansion: J n (·) is the Bessel function of the first kind of order n, n = -N,…,N; The matrix Φ is obtained by truncating the beam expression of the Jacobi-Anger series expansion by ±N orders.
8. The constant beam shape design method according to claim 1, characterized in that: The sound signal picked up by the microphone array is converted into the frequency domain, the frequency domain sound signal of each microphone channel is multiplied by the corresponding weight, superimposed and converted into the time domain to obtain the final output sound signal; or, the playback sound source is converted into the frequency domain, and multiplied with the weight corresponding to each channel of the speaker array and converted into the time domain to obtain the sound played by each speaker unit.
9. An audio device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the constant beam shape design method according to any one of claims 1 to 8 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the constant beam shape design method according to any one of claims 1 to 8.