Beam forming method and system with beam deflection constant capability, storage medium and electronic equipment

Abstract

The invention discloses a beam forming method and system with a beam deflection constant capability, a storage medium and electronic equipment, and the method comprises the steps: obtaining a steering vector of a microphone array according to the directivity of each microphone in the microphone array and a steering vector of an omnidirectional microphone; a linear equation set is obtained based on the steering vector, the undistorted constraint condition and the zero point constraint condition; and based on the linear equation set and the white noise gain constraint condition, constructing an optimization problem for minimizing an error between an actual beam and an ideal beam, and obtaining a weight vector in the actual beam through the optimization problem. The designed beam former has the beam deflection constant capacity, the application field of the beam former is greatly expanded, and the performance indexes such as the white noise gain and the directivity factor of the beam former are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech enhancement, and in particular to a beam forming method, system, storage medium and electronic device with constant beam deflection capability. Background Art

[0002] Speech enhancement technology is an integral component of many far-field pickup devices in noisy environments. Linear differential microphone arrays (LDMA) are able to attenuate ambient noise or interference due to their super-directivity. At the same time, LDMA can form a directional beam pattern, giving the microphone array the ability to pick up sound in a directional manner. Although the gun-type microphones on the market also have the ability to pick up sound in a directional manner, which can pick up sound in a certain direction in a noisy environment, making the picked up sound clearer and less noisy, they have fixed beamforming characteristics and are not tunable. In addition, the cost of designing and producing this type of microphone is relatively high. In contrast, microphone arrays using differential beamforming algorithms can provide a more flexible solution.

[0003] The beam pattern of an LDMA can be quantified by the directivity factor (DF), which is a performance metric that maximizes the ratio of its sensitivity in the direction of the main lobe to its average sensitivity over the entire space. The main lobe direction is the angle of incidence of the desired sound source. The DF of an LDMA beam pattern can increase with the order of the LDMA. However, high-order LDMAs can be very sensitive to the noise generated by each microphone element of the DMA itself, where sensitivity is measured in terms of white noise gain (WNG).

[0004] Currently, in many applications such as hearing aids and Bluetooth headsets, the direction of the sound source can be assumed and the beamformer does not actually need to be designed with beam steering capabilities. However, in many other application areas, such as smartphones, smart computers, smart TVs, etc., a beamformer with beam steering capabilities is required because the sound source position may not be incident along the end-fire direction.

[0005] In summary, designing such an LDMA beamformer with beam steering capability to maximize the acquisition of voice signals is very helpful for realizing various application fields that require directional sound pickup and noise reduction. Summary of the invention

[0006] The object of the present invention is to provide a beamforming method, system, storage medium and electronic device with constant beam deflection capability.

[0007] To achieve the above object, on the one hand, the present invention proposes a beam forming method with constant beam deflection capability, comprising: S1, obtaining a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone; S2, obtaining a linear equation system based on the steering vector and the distortion-free constraint and the zero-point constraint; S3, based on the linear equations and the white noise gain constraint, construct an optimization problem for minimizing the error between the actual beam and the ideal beam, and obtain a weight vector in the actual beam from the optimization problem.

[0008] In a preferred embodiment, the microphone array is a linear differential microphone array formed by combining multiple types of microphones, and the types of microphones include omnidirectional, cardioid and dipole microphones.

[0009] In a preferred embodiment, in S1, the directivity of each microphone is expressed as: ;

[0010] in, For each microphone’s characteristics, is the angle of incidence of the sound, is the number of microphones in the microphone array; The steering vector of the microphone array is expressed as: ; in, , is the microphone response matrix, , is the steering vector of the omnidirectional microphone, is the wave number, ,in is the frequency, is the speed of sound, represents the position coordinates of each microphone, Represents the transpose operation of a vector or matrix.

[0011] In a preferred embodiment, the linear equation system is expressed as: ; in, , is the main beam direction of the beam, represents the transposed conjugate, , For an ideal beam of order N, There are N different zero points in the range, is the weight vector in the actual beam, .

[0012] In a preferred embodiment, in S3, the white noise gain constraint condition is expressed as: ; in, is a white noise gain value that is less than the maximum white noise gain value.

[0013] In a preferred embodiment, in S3, the error between the actual beam and the ideal beam is expressed as: ;

[0014] in, , is the actual beam; is an ideal beam.

[0015] In a preferred embodiment, the optimization problem is expressed as: ; in, .

[0016] On the other hand, the present invention provides a beamforming system with constant beam deflection capability, comprising: A steering vector module, used to obtain a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone; A constraint equation group obtaining module, used for obtaining a linear equation group based on the steering vector and the distortion-free constraint condition and the zero-point constraint condition; The weight acquisition module is used to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation group and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

[0017] In yet another aspect, the present invention provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed, the steps in the above-mentioned beamforming method are executed.

[0018] On the other hand, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps in the above-mentioned beamforming method are executed.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The present invention uses a linear microphone array distribution formed by a combination of multiple types, takes minimizing the error between the actual beam and the ideal beam as the objective function, and combines the above-mentioned zero point constraint, distortion-free constraint and white noise gain constraint to form a constraint condition to design the optimization problem, thereby realizing that the formed beamformer has the ability to deflect the beam, greatly expanding its application neighborhood. In addition, it has been verified that the white noise gain and directivity factor performance indicators of the beamformer designed by the present invention are also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flow chart of a beam forming method with constant beam deflection capability of the present invention; Figure 2 A schematic diagram of a microphone array in an embodiment of the present invention; Figure 3a , 3b , 3c and Figure 3d In one embodiment of the present invention, in a single frequency (frequency is 1000 Hz), the desired direction They are: schematic diagrams of beam comparison between the actual beam formed by the LDDMA beamformer and the ideal beam at 30°, 120°, 210°, and 300°; Figure 4a~Figure 4c A schematic diagram showing a DF performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, a schematic diagram showing a WNG performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, and a schematic diagram showing an error (ERROR) comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method; Figure 5a and Figure 5b Schematic diagrams of beams of an LDDMA beamformer designed by an existing zero-point constraint method and a beam of an LDDMA beamformer designed by the present invention. DETAILED DESCRIPTION

[0021] The specific embodiments of the present invention are described in detail below, but it should be understood that the protection scope of the present invention is not limited by the specific embodiments.

[0022] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising” will be understood to include the stated elements or components but not to exclude other elements or components.

[0023] It should be noted that beamforming is achieved by applying a linear spatial filter The ultimate goal of beamforming, which is achieved by applying microphone observation signals, is to determine the optimal beamformer. , so that the signal from the microphone array matches the desired signal.

[0024] like Figure 1 As shown, a beam forming method with constant beam deflection capability disclosed by the present invention comprises the following steps: S1, obtaining a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone.

[0025] Specifically, in this embodiment, the microphone array is a linear differential microphone array formed by combining multiple types of microphones, such as Figure 2 As shown, the linear differential microphone array has The microphone units may be omnidirectional, cardioid or dipole microphones, etc. In this embodiment, the directivity of each microphone in the microphone array may be expressed as: ; in, For each microphone’s characteristics, such as When , it indicates that the microphone is in cardioid beam mode. , indicating that the microphone is in dipole beam mode. is the angle of incidence of the sound, is the number of microphones in the microphone array. Different from the existing solution in which all microphones in the microphone array are of the same type, the microphone array of the present invention is formed by combining multiple types of microphones, so that the microphone array can achieve beam deflection while maintaining the beam pattern unchanged, that is, it has the ability to maintain constant beam deflection capability.

[0026] The steering vector of the microphone array is obtained by combining the directivity of each microphone in the microphone array with the steering vector of the omnidirectional microphone. Specifically expressed as: ; in, , because of the above The microphone response matrix composed of the directivity of the microphones, , is the steering vector of the omnidirectional microphone, is the wave number, ,in is the frequency, is the speed of sound, represents the position coordinates of each microphone, Represents the transpose operation of a vector or matrix, Is an imaginary unit.

[0027] S2, obtaining a set of linear equations based on the steering vector and the distortion-free constraint and the zero-point constraint.

[0028] Specifically, in this embodiment, the problem of designing N-order differential beamforming for a microphone array is described as a set of linear equations. The linear equations are specifically obtained based on the above-mentioned steering vector and the distortion-free constraint and the zero-point constraint, that is, through the combined design method of the zero-point constraint and the distortion-free constraint, the multiple zero-point positions of the beam are subjected to distortion-free constraints. In this embodiment, the linear equations are specifically expressed in matrix form as follows: .

[0029] in, , is the main beam direction of the beam, represents the transposed conjugate, , For an ideal beam of order N, There are N different zero points in the range, which are specifically expressed as: , is the weight vector in the actual beam, .

[0030] S3, based on the linear equations and the white noise gain constraint, construct an optimization problem for minimizing the error between the actual beam and the ideal beam, and obtain a weight vector in the actual beam from the optimization problem.

[0031] Preferably, the present invention adds a white noise gain constraint condition in the optimization problem. In this embodiment, the white noise gain constraint condition is specifically designed as follows: .

[0032] in, is a white noise gain value that is less than the maximum white noise gain value.

[0033] In addition, the present invention uses the error between the actual beam and the ideal beam as the objective function of the optimization problem. In this embodiment, the error between the actual beam and the ideal beam is specifically expressed as: ; In the formula, , is the actual beam; , is the sound pressure field along The ideal beam of order N in the direction can be written as a vector pattern: , where .

[0034] in It can be calculated by the following formulas: According to the N-order ideal beam , from the above N zero point positions and the main beam direction of the desired beam , the following equation can be listed:

[0035] In the formula, , ; So we can use the following formula: , calculated to obtain .

[0036] error In , , It can be calculated by the following formula: ; ; ; ; .

[0037] Among them, the above The matrix is dimensional matrix, its The elements are: , .

[0038] in is an imaginary unit, is the nth-order Bessel function of the first kind.

[0039] The matrix is dimensional diagonal matrix, its The elements are: .

[0040] for dimensional matrix, its ( m , n ) elements are: , is the Bessel function of the first kind of order 0.

[0041] Therefore, the present invention designs the optimization problem as an optimization problem that takes minimizing the error between the actual beam and the ideal beam as the objective function and combines the above-mentioned zero point constraint, distortion-free constraint and white noise gain constraint to form constraint conditions.

[0042] In this embodiment, the optimization problem is specifically expressed as: .

[0043] By solving the above optimization problem, the weight vector in the actual beam can be calculated: , that is, the final structure of the beamformer is obtained.

[0044] The following is a specific example to illustrate that the LDDMA designed by the present invention has the constant beam deflection capability. In this example, the LDDMA is shown as a 9-unit microphone array, specifically 5 monopoles and 4 dipoles, and the corresponding response matrix The microphone elements are arranged linearly with an inter-element spacing of 0.005 m. The ideal beam pattern desired in this example is a second order ( N =2) Hypercardioid beam pattern (this beam pattern has 2 zero points, ). Figure 3a , 3b , 3c and Figure 3d It shows the expected direction at a single frequency (frequency is 1000Hz) The beam diagrams of the actual beam formed by the LDDMA beamformer and the ideal beam are: 30°, 120°, 210°, and 300°. It can be seen from the figure that the designed LDDMA beamformer has the ability to deflect the beam while keeping the beam pattern constant, and the actual beam formed by it completely or almost completely overlaps with the ideal beam (that is, the error between the two is minimized).

[0045] In addition, the indicators for evaluating the performance of the beamformer usually include white noise gain (WNG), beam pattern and directivity factor (DF). Among them, WNG shows the ability of the beamformer to suppress spatially uncorrelated noise, and is also the most convenient way to evaluate the sensitivity of the beamformer to its certain defects (such as sensor noise, position error, etc.), which is specifically expressed as: The beam pattern shows the directional sensitivity of the beamformer to a plane wave incident on the array from an angle of incidence θ. The directivity factor (DF) is defined as the ratio of the array output response power in the desired steering direction to the average power in the 0° to 360° direction and is expressed as: .

[0046] like Figure 4a~Figure 4cAs shown, there are respectively a schematic diagram of DF performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, a schematic diagram of WNG performance comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method, and a schematic diagram of error comparison between the LDDMA beamformer designed by the present invention and the LDDMA beamformer designed by the existing zero-point constraint method. Figure 4a It can be seen that the DF of the method of the present invention remains approximately constant in the entire evaluation frequency band (i.e., 200 Hz to 8000 Hz), and does not fluctuate like the DF of the zero-point constraint method. The results show that the beam pattern formed by the design of the present invention is more stable. Figure 4b and 4c It can be seen that although the WNG of the method of the present invention is slightly lower than the WNG value of the zero-point constraint method (about 3 dB lower) above 1000 Hz, the error (ERROR) between the beamformer and the ideal beam is more than 15 dB lower than that of the zero-point constraint method. The comparison results show that different from the maximization of white noise gain in the existing zero-point constraint method, the constraint condition designed by the present invention can make fuller use of the degrees of freedom of the array, so that the system can achieve a better balance between the two indicators of robustness (WNG) and the degree of approximation of the beam pattern (ERROR).

[0047] like Figure 5a and Figure 5b As shown in FIG. 1 , the beam diagram of the LDDMA beamformer designed by the existing zero-point constraint method and the beam diagram of the LDDMA beamformer designed by the present invention are shown in FIG. Figure 5a It can be seen that although the existing zero-point constraint method can maintain a constant beam pattern below 5000 Hz, the actual beam will deviate from the ideal beam above 5000 Hz. The actual beam pattern designed by the method of the present invention can remain basically constant in the entire evaluation frequency band (i.e., 200 Hz~8000 Hz), indicating that the beam designed by the present invention has the characteristic of frequency invariance.

[0048] The advantages of the present invention are that the present invention forms a linear microphone array distribution through a combination of multiple types, takes minimizing the error between the actual beam and the ideal beam as the objective function, and combines the above-mentioned zero point constraint, distortion-free constraint and white noise gain constraint to form constraint conditions to design the optimization problem, thereby realizing that the formed beamformer has a constant beam deflection capability, which greatly expands its application neighborhood. In addition, it has been verified that the white noise gain and directivity factor performance indicators of the beamformer designed by the present invention are also improved.

[0049] Corresponding to the above-mentioned beamforming method, the present invention also discloses a beamforming system with constant beam deflection capability, which specifically includes: A steering vector module, used to obtain a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone; A constraint equation group obtaining module, used for obtaining a linear equation group based on the steering vector and the distortion-free constraint condition and the zero-point constraint condition; The weight acquisition module is used to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation group and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

[0050] The working principles of these three modules can refer to the descriptions in the above steps S1 to S3 respectively, and will not be described in detail here.

[0051] On the other hand, the present invention further provides a readable storage medium having a computer program stored thereon, which implements the steps in the beamforming method provided in the above embodiment when the program is executed.

[0052] On the other hand, the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps in the above-mentioned beamforming method are executed.

[0053] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0054] More specific examples (a non-exhaustive list) of readable storage media include the following: an electrical connection with one or more wirings (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the readable storage medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0055] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0056] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A beam forming method with constant beam deflection capability, characterized in that: The method comprises: S1, obtaining a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone; S2, obtaining a linear equation system based on the steering vector and the distortion-free constraint and the zero-point constraint; S3, based on the linear equations and the white noise gain constraint, construct an optimization problem for minimizing the error between the actual beam and the ideal beam, and obtain a weight vector in the actual beam from the optimization problem.

2. A beam forming method with constant beam deflection capability as claimed in claim 1, characterized in that: The microphone array is formed by combining multiple types of microphones, including omnidirectional, cardioid and dipole microphones.

3. A beam forming method with constant beam deflection capability as claimed in claim 1, characterized in that: In S1, the directivity of each microphone is expressed as: ； in, For each microphone’s characteristics, is the angle of incidence of the sound, is the number of microphones in the microphone array; The steering vector of the microphone array is expressed as: ； in, , is the microphone response matrix, , is the steering vector of the omnidirectional microphone, is the wave number, ,in is the frequency, is the speed of sound, represents the position coordinates of each microphone, Represents the transpose operation of a vector or matrix.

4. A beam forming method with constant beam deflection capability as claimed in claim 3, characterized in that: The linear equation system is expressed as: ； in, , is the main beam direction of the beam, represents the transposed conjugate, , For an ideal beam of order N, There are N different zero points in the range, is the weight vector in the actual beam, .

5. A beam forming method with constant beam deflection capability as claimed in claim 4, characterized in that: In S3, the white noise gain constraint condition is expressed as: ； in, is a white noise gain value that is less than the maximum white noise gain value.

6. A beam forming method with constant beam deflection capability as claimed in claim 5, characterized in that: In S3, the error between the actual beam and the ideal beam is expressed as: ； in, , is the actual beam; is an ideal beam.

7. A beam forming method with constant beam deflection capability as claimed in claim 6, characterized in that: The optimization problem is expressed as: ； in, .

8. A beam forming system with constant beam deflection capability, characterized in that: The system comprises: A steering vector module, used to obtain a steering vector of the microphone array according to the directivity of each microphone in the microphone array and the steering vector of the omnidirectional microphone; A constraint equation group obtaining module, used for obtaining a linear equation group based on the steering vector and the distortion-free constraint condition and the zero-point constraint condition; The weight acquisition module is used to construct an optimization problem for minimizing the error between the actual beam and the ideal beam based on the linear equation group and the white noise gain constraint condition, and obtain the weight vector in the actual beam from the optimization problem.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, the steps in the beamforming method according to any one of claims 1 to 7 are executed.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps in the beamforming method according to any one of claims 1 to 7 are executed.

Images