A beamforming method, system, storage medium, and electronic device with a constant beam deflection ability

Abstract

The present invention discloses a beamforming method, system, storage medium and electronic device having a constant beam deflection ability. The method includes obtaining a steering vector of a microphone array according to the directivity of each microphone in the microphone array and the steering vector of an omnidirectional microphone; obtaining a system of linear equations based on the steering vector, the distortionless constraint condition and the null constraint condition; constructing an optimization problem for minimizing the error between an actual beam and an ideal beam based on the system of linear equations and the white noise gain constraint condition, and obtaining a weight vector in the actual beam from the optimization problem. The beamformer designed by the present invention has a constant beam deflection ability, greatly expands its application fields, and also improves performance indexes such as its white noise gain and directivity factor.

Description

Technical Field

[0001] The present invention relates to the technical field of speech enhancement, and particularly to a beamforming method, system, storage medium, and electronic device with a constant beam deflection ability. Background Art

[0002] Speech enhancement technology is an indispensable part of far-field sound pickup devices in many noisy environments. A linear differential microphone array (LDMA) can attenuate ambient noise or interference due to its super directivity. At the same time, LDMA can form a directional beam pattern, enabling the microphone array to have a directional sound pickup ability. Although the gun-shaped microphones on the market also have the ability to pick up sound directionally, which can pick up sound in a certain direction in a noisy environment, making the picked-up sound clearer and less noisy, they have fixed beamforming characteristics and are not tunable. In addition, the cost of designing and producing this type of microphone is relatively high. In contrast, a microphone array using a differential beamforming algorithm can provide a more flexible solution.

[0003] The beam pattern of LDMA can be quantified by the directivity factor (DF), which is a performance index that maximizes the ratio of the sensitivity of the beam pattern in the main lobe direction to its average sensitivity over the entire space. The main lobe direction is the incident angle of the sound source of the desired sound. The DF of the LDMA beam pattern can increase with the order of LDMA. However, a higher-order LDMA may be very sensitive to the noise generated by each microphone element of the DMA itself, where the sensitivity is measured according to the white noise gain (WNG).

[0004] Currently, in many application fields such as hearing aids and Bluetooth headsets, the direction of the sound source can be assumed, and in fact, it is not necessary for the designed beamformer to have the beam deflection ability. However, in many other application fields, such as smart phones, smart computers, smart TVs, etc., a beamformer with the beam deflection ability is required because the sound source position may not be incident along the end-fire direction.

[0005] In summary, designing such a beamformer of LDMA with the beam deflection ability to maximize the collected speech signal is helpful for realizing various application fields that require directional sound pickup and noise reduction. Summary of the Invention

[0006] The purpose of the present invention is to provide a beamforming method, system, storage medium, and electronic device with a constant beam deflection ability.

[0007] To achieve the above purpose, on the one hand, the present invention proposes a beamforming method with a constant beam deflection ability, including:

[0008] S1. Obtain the steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone.

[0009] S2. Based on the steering vector, the distortionless constraint condition, and the null constraint condition, obtain a system of linear equations.

[0010] S3. Based on the system of linear equations and the white noise gain constraint condition, construct an optimization problem that minimizes the error between the actual beam and the ideal beam, and obtain the weight vector in the actual beam from the optimization problem.

[0011] In a preferred embodiment, the microphone array is a linear differential microphone array formed by combining multiple types of microphones, and the types of microphones include omnidirectional, cardioid, and dipole microphones.

[0012] In a preferred embodiment, in S1, the directivity of each microphone is expressed as:

[0013] ;

[0014] where is the characteristic of each microphone, is the sound incident angle, is the number of microphones in the microphone array;

[0015] The steering vector of the microphone array is expressed as:

[0016] ;

[0017] where , is the microphone response matrix, , is the steering vector of the omnidirectional microphone, is the wave number, , where is the frequency, is the speed of sound, represents the position coordinates of each microphone, represents the transpose operation of a vector or matrix.

[0018] In a preferred embodiment, the system of linear equations is expressed as:

[0019] ;

[0020] where , is the main beam direction of the beam, represents the conjugate transpose, , is the Nth ideal beam at There are N different zeros within the range, is the weight vector in the actual beam, .

[0021] In a preferred embodiment, in S3, the white noise gain constraint condition is expressed as:

[0022] ;

[0023] where, is the white noise gain value less than the maximum value of the white noise gain.

[0024] In a preferred embodiment, in S3, the error between the actual beam and the ideal beam is expressed as:

[0025] ;

[0026] where, , is the actual beam; is the ideal beam.

[0027] In a preferred embodiment, the optimization problem is expressed as:

[0028] ;

[0029] where, .

[0030] On the other hand, the present invention proposes a beamforming system with a constant beam deflection ability, including:

[0031] A steering vector module, configured to obtain the steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone;

[0032] A constraint equation set obtaining module, configured to obtain a linear equation set based on the steering vector, the distortionless constraint condition, and the null constraint condition;

[0033] A weight obtaining module, configured to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation set and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

[0034] On yet another aspect, the present invention proposes a readable storage medium, in which a computer program is stored, and when the computer program is run, it executes the steps in the above beamforming method.

[0035] On still another aspect, the present invention proposes an electronic device, including a memory and a processor, where a computer program is stored in the memory, and when the computer program is run by the processor, it executes the steps in the above beamforming method.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] Through the linear microphone array distribution formed by combining multiple types, taking the minimization of the error between the actual beam and the ideal beam as the objective function, and combining the above zero-point constraint, distortionless constraint, and white noise gain constraint conditions to form a constraint condition to design an optimization problem, the formed beamformer has the beam deflection ability, greatly expanding its application field. In addition, it has been verified that the performance indicators of the white noise gain and directivity factor of the beamformer designed by the present invention have also been improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic flow chart of the beamforming method with a constant beam deflection ability according to the present invention;

[0039] Figure 2 is a schematic diagram of the microphone array in an embodiment of the present invention;

[0040] Figure 3a 、 3b 、3c and Figure 3d are the schematic diagrams of the beam comparison between the actual beam and the ideal beam formed by the LDDMA beamformer at a single frequency (frequency 1000 Hz) and desired directions which are: 30°, 120°, 210°, 300° respectively in an embodiment of the present invention;

[0041] Figures 4a to 4c are respectively the schematic diagrams of the DF performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, the schematic diagram of the WNG performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, and the schematic diagram of the error (ERROR) comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method;

[0042] Figure 5a and Figure 5b are respectively the beam diagrams of the LDDMA beamformer designed by the existing zero-point constraint method and the beam diagram of the LDDMA beamformer designed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The following is a detailed description of the specific embodiments of the present invention, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0044] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or variations thereof such as "comprises" or "comprising" shall be understood to include the stated element or component, without excluding other elements or other components.

[0045] It should be noted that beamforming is achieved by applying a linear spatial filter to the microphone observation signals, and the ultimate goal of beamforming is to determine the optimal beamformer so that the signals from the microphone array match the desired signals.

[0046] As Figure 1 shown, a beamforming method with a constant beam deflection ability disclosed by the present invention includes the following steps:

[0047] S1. Obtain the steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone.

[0048] Specifically, in this embodiment, the microphone array is a linear differential microphone array formed by combining multiple types of microphones. As Figure 2 shown, the linear differential microphone array has microphone units. Among them, the types of microphones can be omnidirectional, cardioid or dipole microphones, etc. In this embodiment, the directivity of each microphone in the microphone array can be expressed as:

[0049] ;

[0050] where is the characteristic of each microphone. For example, when , it means the microphone is in a cardioid beam pattern, and when , it means the microphone is in a dipole beam pattern. is the sound incident angle, is the number of microphones in the microphone array. Different from the scheme where all microphones in the existing microphone array have the same type, the microphone array of the present invention is formed by combining multiple types of microphones, enabling the microphone array to maintain the beam pattern unchanged while achieving beam deflection, that is, having the ability of constant beam deflection.

[0051] By combining the directivity of each microphone in the microphone array with the steering vector of the omnidirectional microphone, the steering vector of the microphone array is obtained. In this embodiment, the steering vector of the microphone array is specifically expressed as:

[0052] ;

[0053] Among them, , is the microphone response matrix composed of the directivities of the above microphones, , is the steering vector of the omnidirectional microphone, is the wave number, , where is the frequency, is the speed of sound, represents the position coordinates of each microphone, represents the transpose operation of a vector or matrix, is the imaginary unit.

[0054] S2, based on the said steering vector, the distortionless constraint condition and the null constraint condition, a linear equation system is obtained.

[0055] Specifically, in this embodiment, the problem of designing an N - order differential beamforming for a microphone array is described as a linear equation system. This linear equation system is specifically obtained based on the above - mentioned steering vector, the distortionless constraint condition and the null constraint condition, that is, through the combined design method of null constraint and distortionless constraint, the distortionless constraint is imposed on multiple null positions of the beam. In this embodiment, the linear equation system is specifically represented in matrix form as:

[0056] .

[0057] Among them, , is the main beam direction of the beam, represents the conjugate transpose, , is the N different nulls existing in the N - order ideal beam within the range, specifically represented as: , is the weight vector in the actual beam, .

[0058] S3, based on the said linear equation system and the white noise gain constraint condition, an optimization problem of minimizing the error between the actual beam and the ideal beam is constructed, and the weight vector in the actual beam is obtained from the said optimization problem.

[0059] Preferably, the present invention adds a white noise gain constraint condition to the optimization problem. In this embodiment, the white noise gain constraint condition is specifically designed as:

[0060] .

[0061] Among them, is the white noise gain value less than the maximum value of the white noise gain.

[0062] In addition, the present invention takes the error between the actual beam and the ideal beam as the objective function of the optimization problem. In this embodiment, the error between the actual beam and the ideal beam is specifically expressed as:

[0063] ;

[0064] In the formula, , is the actual beam; , is the N - th order ideal beam of the sound pressure field along the direction, and written in vector form as: , in the formula, .

[0065] Among them can be calculated by the following several formulas:

[0066] Specifically, according to the N - th order ideal beam , from the above N zero - point positions and the main beam direction of the expected beam, the following equations can be listed:

[0067]

[0068] In the formula, , ; thus, from the following formula:

[0069] , can be calculated.

[0070] The error in , , can be calculated by the following formulas:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] .

[0076] Among them, the above matrix is a - dimensional matrix, and its - th element is:

[0077] ,

[0078] .

[0079] Wherein is the imaginary unit, is the Bessel function of the first kind of order n.

[0080] The matrix is a diagonal matrix of dimension whose

[0081] -th element is:

[0082] is a matrix of dimension m , n ) whose element is:

[0083] , being the Bessel function of the first kind of order 0.

[0084] Therefore, the optimization problem of the present invention is designed to minimize the error between the actual beam and the ideal beam as the objective function, and to form a constraint condition by combining the above zero-point constraint, distortionless constraint and white noise gain constraint conditions.

[0085] In this embodiment, the optimization problem is specifically expressed as:

[0086] .

[0087] By solving the above optimization problem, the weight vector in the actual beam can be calculated, that is, the finally constructed beamformer is obtained.

[0088] The following uses a specific embodiment to illustrate that the LDDMA designed by the present invention has the ability of constant beam deflection. In this example, the LDDMA is shown as a 9-element microphone array, specifically 5 monopoles and 4 dipoles, and the corresponding response matrix , and these microphone elements are linearly arranged with an element spacing of 0.005 m. The desired ideal beam pattern in this example is a second-order ( N =2) supercardioid beam pattern (this beam pattern has 2 zero points, which are ). Figure 3a , 3b , 3c and Figure 3d illustrate at a single frequency (frequency 1000 Hz), the desired direction They are: the beam patterns of the actual beam and the ideal beam formed by the 30°, 120°, 210°, and 300° LDDMA beamformers. It can be seen from the figure that while the designed LDDMA beamformer has the ability of beam deflection, the beam pattern remains constant, and the actual beam formed by it completely or basically coincides with the ideal beam (i.e., the error between the two is minimized).

[0089] In addition, the indicators for evaluating the performance of a beamformer usually include the white noise gain (WNG), beam pattern, and directivity factor (DF). Among them, WNG shows the ability of the beamformer to suppress spatially uncorrelated noise and is also the most convenient method for evaluating the sensitivity of the beamformer to some of its defects (such as sensor noise, position error, etc.). It is specifically expressed as: . The beam pattern shows the direction sensitivity of the beamformer to the plane wave incident on the array from the incident angle θ. The directivity factor (DF) is defined as the ratio between the array output response power in the desired steering direction and the average power in the direction from 0° to 360°. It is specifically expressed as: .

[0090] As Figures 4a to 4c shown, they are respectively the schematic diagram of the DF performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing null constraint method, the schematic diagram of the WNG performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing null constraint method, and the schematic diagram of the error comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing null constraint method. It can be seen from Figure 4a that the value of the DF of the method of the present invention remains approximately constant throughout the evaluation frequency band (i.e., 200 Hz to 8000 Hz) and does not fluctuate like the DF of the null constraint method, indicating that the beam pattern formed by the design of the present invention is more stable. It can be seen from Figure 4b and 4c that although above 1000 Hz, the WNG of the method of the present invention is slightly lower than the WNG value of the null constraint method (about 3 dB lower), the error (ERROR) between the beamformer and the ideal beam is more than 15 dB lower than that of the null constraint method. The comparison results show that different from the method of maximizing the white noise gain in the existing null constraint method, the designed constraint condition of the present invention can make more full use of the degrees of freedom of the array, enabling the system to achieve a better balance between the two indicators of robustness (WNG) and the approximation degree of the beam pattern (ERROR).

[0091] As Figure 5a and Figure 5bAs shown, there are respectively the beam pattern of the LDDMA beamformer designed by the existing zero-point constraint method and the beam schematic diagram of the LDDMA beamformer designed by the present invention. From Figure 5a It can be seen that although the existing zero-point constraint method can maintain a constant beam pattern below 5000 Hz, the actual beam will deviate from the ideal beam above 5000 Hz. However, the actual beam pattern designed by the method of the present invention can remain basically constant throughout the evaluation frequency band range (i.e., 200 Hz to 8000 Hz), indicating that the beam designed by the present invention has the characteristic of frequency invariance.

[0092] The advantages of the present invention are as follows: The present invention designs an optimization problem by forming a linear microphone array distribution through a combination of various types, taking the minimization of the error between the actual beam and the ideal beam as the objective function, and combining the above zero-point constraint, distortionless constraint, and white noise gain constraint conditions to form the constraint conditions. As a result, the formed beamformer has the ability to maintain a constant beam deflection, greatly expanding its application field. In addition, it has been verified that the performance indicators of the white noise gain and directivity factor of the beamformer designed by the present invention have also been improved.

[0093] Corresponding to the above beamforming method, the present invention also discloses a beamforming system with the ability to maintain a constant beam deflection, specifically including:

[0094] A steering vector module, configured to obtain the steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone;

[0095] A constraint equation set obtaining module, configured to obtain a linear equation set based on the steering vector, the distortionless constraint condition, and the zero-point constraint condition;

[0096] A weight obtaining module, configured to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation set and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

[0097] Among them, the working principles of these three modules can be respectively referred to the descriptions in the above steps S1 to S3, and will not be elaborated here.

[0098] On the other hand, the present invention also provides a readable storage medium, on which a computer program is stored, and when the program is run, it implements the steps in the beamforming method provided in the above embodiment.

[0099] On yet another aspect, the present invention also provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is run by the processor, it executes the steps in the above beamforming method.

[0100] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered as a definable sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0101] More specific examples (non-exhaustive list) of readable storage media include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the readable storage media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other appropriate processing as necessary, and then stored in a computer memory.

[0102] It should be understood that the various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well-known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0103] The foregoing description of the specific exemplary embodiments of the present invention is for purposes of illustration and exemplification. These descriptions are not intended to limit the present invention to the precise forms disclosed, and it is obvious that many changes and variations are possible in light of the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical applications, so that those skilled in the art can implement and utilize the various different exemplary embodiments of the present invention, as well as various different selections and changes. The scope of the present invention is intended to be defined by the claims and their equivalents.

Claims

1. A beam forming method with constant beam deflection capability, characterized in that: The method comprises: S1, obtaining a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone, wherein the microphone array is formed by combining multiple types of microphones; S2, obtaining a linear equation system based on the steering vector and the distortion-free constraint and the zero-point constraint; S3, based on the linear equations and the white noise gain constraint, construct an optimization problem for minimizing the error between the actual beam and the ideal beam, and obtain a weight vector in the actual beam from the optimization problem.

2. A beam forming method with constant beam deflection capability as claimed in claim 1, characterized in that: The types of microphones include omnidirectional, cardioid and dipole microphones.

3. A beam forming method with constant beam deflection capability as claimed in claim 1, characterized in that: In S1, the directivity of each microphone is expressed as: ； in, For each microphone’s characteristics, is the angle of incidence of the sound, is the number of microphones in the microphone array; The steering vector of the microphone array is expressed as: ； in, , is the microphone response matrix, , is the steering vector of the omnidirectional microphone, is the wave number, ,in is the frequency, is the speed of sound, represents the position coordinates of each microphone, Represents the transpose operation of a vector or matrix.

4. A beam forming method with constant beam deflection capability as claimed in claim 3, characterized in that: The linear equation system is expressed as: ； in, , is the main beam direction of the beam, represents the transposed conjugate, , For an ideal beam of order N, There are N different zero points in the range, is the weight vector in the actual beam, .

5. A beam forming method with constant beam deflection capability as claimed in claim 4, characterized in that: In S3, the white noise gain constraint condition is expressed as: ； in, is a white noise gain value that is less than the maximum white noise gain value.

6. A beam forming method with constant beam deflection capability as claimed in claim 5, characterized in that: In S3, the error between the actual beam and the ideal beam is expressed as: ； in, , is the actual beam; is an ideal beam.

7. A beam forming method with constant beam deflection capability as claimed in claim 6, characterized in that: The optimization problem is expressed as: ； in, .

8. A beam forming system with constant beam deflection capability, characterized in that: The system comprises: A steering vector module, used to obtain a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone, wherein the microphone array is formed by combining multiple types of microphones; A constraint equation group obtaining module, used for obtaining a linear equation group based on the steering vector and the distortion-free constraint condition and the zero-point constraint condition; The weight acquisition module is used to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation group and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, the steps in the beamforming method according to any one of claims 1 to 7 are executed.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps in the beamforming method according to any one of claims 1 to 7 are executed.

Images