Beam forming method, apparatus and device

Abstract

The application discloses a beam forming method, device and equipment. The method comprises the following steps: obtaining a target signal incident direction, a target signal incident elevation angle, microphone pointing direction information and microphone array shape information; determining a response vector of the microphone array according to the microphone pointing direction information, the target signal incident direction and the target signal incident elevation angle; determining a first steering vector of the microphone array according to the microphone array shape information, the target signal incident direction and the target signal incident elevation angle; and determining a weight vector of a super-directive beam former according to the response vector and the first steering vector. The processing method makes the super-directive beam forming based on the directional microphone array, and increases the amplitude response obtained based on the microphone pointing information in the super-directive beam forming, so that higher white noise gain can be obtained in a low frequency band, higher directivity factor can be obtained in a high frequency band, and the robustness of the super-directive beam forming is improved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, specifically to beamforming methods and apparatus, and sound pickup devices. Background Technology

[0002] Internet technology has changed people's communication tools, and cloud-based audio and video conferencing systems are gradually becoming more widespread. In noisy and reverberant environments, audio and video conferencing terminals use beamforming technology to achieve high-quality sound pickup; therefore, beamforming in video conferencing environments has always been a research hotspot.

[0003] Microphone array-based beamforming can enhance the desired direction of a target signal and suppress interference from undesired directions. A typical beamforming method is super-directional beamforming using a uniform circular array of omnidirectional microphones. White noise gain (WNG) and directivity factor (DF) are two important metrics for evaluating beamforming performance. White noise gain measures the robustness of the beamformer, while the directivity factor assesses array performance in the presence of reverberation.

[0004] However, in the process of realizing this invention, the inventors found that the existing solutions have at least the following problems: the beam pattern changes with frequency, specifically: the white noise gain is low at low frequencies and the directivity factor decreases at high frequencies. Therefore, super-directional beamforming based on omnidirectional microphone arrays has the problem of low robustness. Summary of the Invention

[0005] This application provides a beamforming method to address the problem of low robustness in super-directional beamforming based on omnidirectional microphone arrays. This application also provides a beamforming apparatus and a sound pickup device.

[0006] This application provides a beamforming method for a sound pickup device, the sound pickup device including a directional microphone array, the method comprising:

[0007] Acquire the target signal incident direction and target signal incident elevation angle, microphone pointing direction information, and microphone array shape information;

[0008] The response vector of the microphone array is determined based on the microphone pointing direction information, the incident direction of the target signal, and the incident elevation angle of the target signal.

[0009] Based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal, determine the first steering vector of the microphone array;

[0010] The weight vector of the super-directing beamformer is determined based on the response vector and the first steering vector.

[0011] Optionally, the directional microphone array includes a ring array;

[0012] The microphone pointing direction information is obtained in the following way:

[0013] Obtain the first angle between the microphone and the horizontal direction and the second angle between the microphone orientation and the diameter direction;

[0014] The sum of the first included angle and the second included angle is used as the microphone pointing direction information.

[0015] Optionally, determining the response vector of the microphone array based on the microphone pointing direction information, the incident direction of the target signal, and the incident elevation angle of the target signal includes:

[0016] Obtain the angle between the incident direction of the target signal and the pointing direction of the microphone;

[0017] The amplitude response of the directional microphone is obtained based on the included angle and the incident elevation angle of the target signal.

[0018] Optionally, the directional microphone is a first-order directional microphone;

[0019] The step of determining the response vector of the microphone array based on the microphone pointing direction information, the incident direction of the target signal, and the incident elevation angle of the target signal includes:

[0020] The response vector of the microphone array is determined based on the microphone pointing direction information, the incident direction of the target signal, the incident elevation angle of the target signal, and the coefficients of the first-order directional microphone.

[0021] Optional, also includes:

[0022] Set different coefficients for first-order directional microphones for different directional microphones.

[0023] Optionally, the directional microphone array can be a circular array;

[0024] The step of determining the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal includes:

[0025] Obtain the third angle between the incident direction of the target signal and the first angle between the microphone and the microphone, where the first angle is the angle between the microphone and the horizontal direction of the circular array.

[0026] Based on the radius information of the circular array, the third included angle, and the incident elevation angle of the target signal, the first guide vector of the directional microphone is obtained.

[0027] Optionally, the directional microphone array can be a linear array;

[0028] The step of determining the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal includes:

[0029] Based on the microphone array shape information, the distance values ​​between each microphone and the reference microphone are obtained. The reference microphone can be the microphone that is the first microphone in the linear array from left to right.

[0030] The first guiding vector is obtained based on the distance value, the incident direction of the target signal, and the incident elevation angle of the target signal.

[0031] Optionally, determining the weight vector of the super-directing beamformer based on the response vector and the first steering vector includes:

[0032] The second steering vector is determined based on the response vector and the first steering vector;

[0033] Determine the spherical noise field matrix based on the second guiding vector;

[0034] The weight vector is determined based on the spherical noise field matrix and the second steering vector.

[0035] This application also provides a beamforming apparatus for a sound pickup device, the sound pickup device including a directional microphone array, characterized in that it includes:

[0036] The parameter acquisition unit is used to acquire the incident direction and elevation angle of the target signal, microphone pointing direction information, and microphone array shape information;

[0037] The response vector determination unit is used to determine the response vector of the microphone array based on the microphone pointing direction information, the incident direction of the target signal, and the incident elevation angle of the target signal.

[0038] The first steering vector determination unit is used to determine the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal and the incident elevation angle of the target signal;

[0039] The weight vector determination unit is used to determine the weight vector of the super-directing beamformer based on the response vector and the first steering vector.

[0040] This application also provides a sound pickup device, including:

[0041] speaker;

[0042] Directional microphone array;

[0043] Processor; and

[0044] The memory is used to store the program that implements the above method, and the terminal is powered on and runs the program of the method through the processor.

[0045] This application also provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the various methods described above.

[0046] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to perform the various methods described above.

[0047] Compared with the prior art, this application has the following advantages:

[0048] The beamforming method provided in this application acquires the incident direction and elevation angle of the target signal, microphone pointing direction information, and microphone array shape information; determines the response vector of the microphone array based on the microphone pointing direction information, the incident direction and elevation angle of the target signal; determines the first steering vector of the microphone array based on the microphone array shape information, the incident direction and elevation angle of the target signal; and determines the weight vector of the super-directional beamformer based on the response vector and the first steering vector. This approach enables super-directional beamforming based on a directional microphone array, adding a degree of freedom to the super-directional beamforming process—the amplitude response obtained based on the microphone pointing information. This results in higher white noise gain in the low-frequency band and a higher directivity factor in the high-frequency band, with a more frequency-invariant beam pattern; therefore, it effectively improves the robustness of super-directional beamforming. Attached Figure Description

[0049] Figure 1 A flowchart illustrating an embodiment of the beamforming method provided in this application;

[0050] Figure 2 A schematic diagram of the sound source signal direction of an embodiment of the beamforming method provided in this application;

[0051] Figure 3 A schematic diagram of a uniform ring array of an embodiment of the beamforming method provided in this application;

[0052] Figure 4 A schematic diagram of a non-uniform ring array of an embodiment of the beamforming method provided in this application;

[0053] Figure 5 This application provides a schematic diagram of the structure of an embodiment of the beamforming apparatus. Detailed Implementation

[0054] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0055] This application provides a beamforming method and apparatus, and a conference terminal. The various solutions are described in detail in the following embodiments.

[0056] First Embodiment

[0057] Please refer to Figure 1 This is a flowchart illustrating an embodiment of the beamforming method of this application. In this embodiment, the method may include the following steps:

[0058] Step S101: Obtain the incident direction and elevation angle of the target signal, the pointing direction of the microphone, and the shape information of the microphone array.

[0059] The method provided in this application is used in a sound pickup device, which includes, but is not limited to, audio and video conferencing terminals, such as speakerphones. The sound pickup device includes a directional microphone array, and the directional microphones include, but are not limited to, cardioid, supercardioid, shotgun, and bidirectional types. The microphone array can be, for example,... Figure 2 The circular array shown can also be an array of other geometric shapes, such as a linear array, a square array, a triangular array, or an array of irregular geometric shapes.

[0060] Beamforming is a spatial filtering algorithm implemented based on microphone arrays. Spatial filtering involves defining a target direction; signals within that direction are picked up, while signals outside that direction are suppressed. Therefore, based on beamforming, a microphone array can pick up sound within a specific directional range while suppressing sound from directions outside that range.

[0061] In this embodiment, the signal direction is a direction in three-dimensional space, and the target direction includes the incident direction of the target signal and the incident elevation angle of the target signal. For example... Figure 2 As shown, the incident direction θ of the target signal d The value range is 0-360 degrees, and the target signal incident elevation angle is... It is the angle between the projection of the incident signal x(t) onto the microphone array plane and the horizontal direction x. The super-directional beamformer formed using the method provided in this application's embodiments is directed towards a specific direction. The signal is not distorted, and the microphone array can pick up sound within a specific direction; the signal outside the specific direction is distorted, and the microphone array can suppress sound from that direction. θ d and These are the design parameters of the beamformer, which determine the properties of the super-directional beamformer, and can be preset.

[0062] The numerous reflections of sound signals in space create a near-spherical isotropic noise field. Superdirective beamforming (SBF) utilizes an isotropic scattered noise field matrix to suppress the influence of noise from all directions on the target signal, outputting an array gain signal with higher "directivity" than conventional beamforming, thereby improving noise reduction performance. Existing superdirective beamforming methods use a uniform circular array of omnidirectional microphones, utilizing only the array shape information of the omnidirectional microphone array. However, the method provided in this application not only utilizes the microphone array shape information but also incorporates the acoustic characteristics of the directional microphones, which are related to the microphone's pointing direction, into the beamforming calculation, thus requiring the acquisition of microphone pointing direction information.

[0063] Directional microphones have different sensitivities to sound from different directions. The microphone's pointing direction information is related to the microphone's orientation and the geometry of the array. The microphone array can be a circular array, where the pointing direction information can be obtained as follows: acquire the first angle between the microphone and the horizontal direction and the second angle between the microphone's orientation and the diameter direction; the second angle is also called the off-axis angle of the directional microphone. The sum of the first and second angles is the microphone's pointing direction information. The microphone array can also be a linear array, where the pointing direction information is the angle between the microphone's orientation and the end-fire direction of the microphone array.

[0064] like Figure 3 As shown, a directional microphone array can be a uniformly oriented and evenly arranged circular array, simply referred to as a uniform ring array. The array shape information includes the radius r of the circular array and the angle between the microphones and the horizontal direction of the microphone array. Etc., for example, the first angle between the m-th microphone and the horizontal direction is... Because the directional microphone array is a uniformly arranged circular array, the angle between two adjacent directional microphones is... Where M represents the number of microphones in the directional microphone array. In practical implementation, the formula can be used. Perform the calculation, where m represents the m-th microphone. This indicates the angle between the first microphone and the horizontal direction of the microphone array. The angle can be greater than or equal to 0. In this case, if the orientation of each microphone is consistent with the diameter of the microphone array, the microphone pointing direction information is the angle between the microphone and the horizontal direction of the microphone array. If the angle between the orientation of each microphone and the diameter of the microphone array is the same, the microphone pointing direction information is the sum of the angle between the microphone and the horizontal direction of the microphone array and the angle between the microphone orientation and the diameter of the microphone array.

[0065] like Figure 4 As shown, a directional microphone array can also be a circular array that is non-uniformly directional and non-uniformly arranged. Figure 4 This shows the direction of the microphone relative to the center of the array, that is, the second angle between the microphone and the diameter direction of the circular array. For example, the angle between the first microphone and the diameter direction is α1, and the angle between the m-th microphone and the diameter direction is α. m The angle between the Mth microphone and the diameter direction is α. M In this case, the pointing direction of the m-th microphone can be the first angle between the microphone and the horizontal direction. The second angle α between the microphone and the diameter direction m The sum of, i.e.

[0066] The array shape information is related to the number of microphones and the geometry of the array. Taking a linear array as an example, the array shape information may include the distance between microphones; the distances between different adjacent microphones may be the same or different. Taking a circular array as an example, the array shape information may include the radius, the angle between the microphones and the horizontal direction of the microphone array, and other information.

[0067] Step S103: Determine the response vector of the microphone array based on the microphone pointing direction information, the incident direction of the target signal, and the incident elevation angle of the target signal.

[0068] The method provided in this application incorporates the acoustic characteristics of the directional microphone device and the pitch angle information of the incident signal in three-dimensional space into the beamforming calculation. This step can determine the amplitude response of the directional microphone based on the microphone pointing direction, the incident direction of the target signal, and the incident pitch angle of the target signal.

[0069] When a directional microphone acquires a signal, for each microphone itself, after sound is acquired from different directions, the directional microphone will have an amplitude response caused by the angle between the pointing direction of the directional microphone and the incident direction of the signal. If different microphones point in different directions, then the microphones will have different amplitude responses to the speech signal from a certain direction due to the different angles. In this embodiment, the amplitude response is caused by the amplitude difference brought about by the directional microphone itself, which is related to the pointing direction of the directional microphone and is not related to the positional differences between the microphones. The response vector includes the amplitude responses corresponding to multiple microphones.

[0070] In specific implementation, step S103 can be achieved as follows: obtain the angle between the incident direction of the target signal and the pointing direction of the microphone, which is also called the off-axis angle of the incident signal relative to the microphone; obtain the amplitude response of the directional microphone based on the angle and the incident elevation angle of the target signal.

[0071] In one example, considering the size or price of a directional microphone, a first-order directional microphone can be used; accordingly, step S103 can be implemented as follows: determine the response vector of the microphone array based on the microphone pointing direction information, the target signal incident direction and the target signal incident elevation angle, and the coefficients of the first-order directional microphone. The coefficients of the first-order directional microphone can be represented by p, where p = 0 forms a dipole beam pattern, p = 0.5 forms a cardioid beam pattern, and p = 1 is an omnidirectional microphone.

[0072] When using a first-order directional microphone, the step of obtaining the amplitude response of the directional microphone based on the included angle and the incident elevation angle of the target signal can be carried out as follows: obtaining the cosine value of the included angle; obtaining the sine value of the incident elevation angle of the target signal; and obtaining the amplitude response of the directional microphone based on the cosine and sine values. Specifically, the amplitude response of a first-order directional microphone can be expressed by the following formula:

[0073]

[0074] In this formula, Let θ be the amplitude response of the m-th directional microphone; θ is the signal incident direction, ranging from 0 to 360 degrees. α is the angle between the m-th directional microphone and the horizontal direction of the microphone array. m Let be the angle between the orientation of the m-th directional microphone and the diameter of the circular array; Let m be the pointing direction of the m-th directional microphone. Let be the elevation angle of the incident signal, and p be the coefficient of a first-order directional microphone.

[0075] In one example, the method may further include the step of setting different coefficients p for the first-order directional microphones for different directional microphones. By controlling p individually for each directional microphone, such as p = 1, 0.5, or 0, more degrees of freedom can be introduced, making it suitable for some special cases.

[0076] In this design, p=0 forms a dipole beamforming pattern, p=0.5 forms a cardioid beamforming pattern, and p=1 is an omnidirectional microphone. These three patterns correspond to three different microphone types, differing primarily in their acoustic characteristics. The core of the method provided in this application is to utilize the acoustic characteristics of the microphone when designing a superdirectional beamformer. When the microphone's acoustic characteristics differ, the designed beamforming will also differ. Specifically, microphones based on dipole (p=0) or cardioid (p=0.5) patterns will provide a significant improvement in WNG compared to omnidirectional microphones (p=1).

[0077] It should be noted that the method provided in this application is not limited to a first-order directional microphone, but can also achieve higher-order directional microphones. When achieving higher-order directional microphones, the amplitude response matrix of the directional microphone is a higher-order response matrix.

[0078] Step S105: Determine the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal.

[0079] The first steering vector represents the relationship between the phase difference between different microphones in the array and the relationship between the directions of the incoming sound waves. The first steering vector describes the spatial phase difference, and its structure is related to the relative positions of the array elements.

[0080] The method provided in this application incorporates the elevation angle information of the incident signal in three-dimensional space into the beamforming calculation. This step determines the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal. This first steering vector is that of a traditional omnidirectional microphone array.

[0081] In one example, the directional microphone array is a uniformly arranged ring array, and step S105 may include the following sub-steps: obtaining a third angle between the incident direction of the target signal and a first angle between the microphone and the first angle, wherein the first angle is the angle between the microphone and the horizontal direction of the ring array; obtaining a first guide vector of the directional microphone based on the radius information of the ring array, the third angle and the incident elevation angle of the target signal.

[0082] In practical implementation, the phase difference of the m-th directional microphone in direction θ can be expressed by the following formula:

[0083]

[0084] In this formula, θ represents the incident direction of the signal, and its value ranges from 0 to 360 degrees. The imaginary unit is represented by ω = 2πf, which represents the angular frequency, and f represents the time frequency (also known as the timing frequency or flicker frequency, or simply time frequency). Let be the first angle between the m-th directional microphone and the horizontal direction of the microphone array; It is the third angle between the signal incident direction and the first angle of the m-th directional microphone; θ is the elevation angle of the incident signal; r is the radius of the circular array.

[0085] In another example, the directional microphone array is a linear array, and step S105 can be implemented as follows: based on the shape information of the microphone array, obtain the distance value between each microphone and the reference microphone, wherein the reference microphone may be the microphone at the first position of the linear array from left to right; based on the distance value, the incident direction of the target signal and the incident elevation angle of the target signal, obtain the first guiding vector.

[0086] In practical implementation, the phase difference of the m-th directional microphone in direction θ can be expressed by the following formula:

[0087]

[0088] In this formula, θ represents the incident direction of the signal. The imaginary unit is represented by ω = 2πf, which represents the angular frequency and f represents the time frequency (also known as the timing frequency or flashing frequency, or simply time frequency); δ is the distance between adjacent microphones, and (m-1)δ represents the distance between the m-th microphone and the first microphone.

[0089] It should be noted that, in specific implementation, step S105 can be executed first to calculate the first guiding vector; then step S103 can be executed to calculate the above-mentioned response vector.

[0090] Step S107: Determine the weight vector of the super-directing beamformer based on the response vector and the first steering vector.

[0091] Multichannel beamforming is a process of adjusting and summing the weighting factor coefficients of the signals received by each microphone channel. The beam pattern ensures that signals within the target direction range are picked up, while signals outside the target direction range are suppressed. The beam pattern is the product of the weight vector and the steering vector of the beamformer. The beam pattern can be expressed by the following formula:

[0092]

[0093] h(ω)=[H1(ω)H2(ω)…H M (ω)] T

[0094] The superscript H indicates conjugate transpose. The guide vector is represented by h(ω); the weight vector for beamforming is represented by θ. d Indicates the incident direction of the target signal. The pitch angle represents the incident elevation angle of the target signal, and M represents the number of microphones. This formula means: for a specific direction... The signal is undistorted, and the microphone array can pick up sound within a specific directional range, with a product of 1; for signals outside the specific direction that are distorted, the microphone array can suppress sound outside the specific direction, with a product of less than 1. h(ω) represents the beamforming weight vector, which is the parameter to be determined.

[0095] In existing technologies, the beam pattern is the product of the beamformer's weight vector and the first steering vector. For each array element, assuming the signal amplitude is the same at any instant, different weight vectors are used for weighting, only adjusting the signal phase and not changing the amplitude.

[0096] The method provided in this application embodiment includes the directional characteristics of the microphone, incorporating these acoustic parameters into the calculation of superdirective beamforming. This step adds a degree of freedom to the existing superdirective beamforming (SBF), namely, the amplitude response obtained based on the microphone directional information. Based on the response vector and the steering vector (i.e., the first steering vector) of a conventional omnidirectional microphone array, the weight vector of the beamformer is determined to form the aforementioned beam pattern.

[0097] In one example, step S107 may include the following sub-steps:

[0098] Step S1071: Determine the second steering vector based on the response vector and the first steering vector.

[0099] The second steering vector is related not only to the phase difference but also to the amplitude response caused by the microphone's pointing. The second steering vector can be represented in the following form:

[0100]

[0101] Where M represents the number of microphones in the microphone array.

[0102] In practical implementation, if the microphone array is a circular array of directional microphones, and a first-order directional microphone is used, then the elements in the second steering vector... The following formula can be used for calculation:

[0103]

[0104] This formula yields the following relationship: the second steering vector of the microphone array can be the fundamental product of the response vector and the first steering vector, which can be expressed as:

[0105]

[0106] in,

[0107]

[0108]

[0109]

[0110] In practical implementation, if the microphone array is a linear array composed of non-uniformly pointing directional microphones, then the element d in the second steering vector... m (ω,θ) can be calculated using the following formula:

[0111]

[0112] This formula yields the following relationship: the second steering vector of the microphone array can be the fundamental product of the response vector and the first steering vector, which can be expressed as:

[0113]

[0114] in,

[0115]

[0116]

[0117] Step S1073: Determine the spherical noise field matrix based on the second guiding vector.

[0118] The numerous reflections of sound signals in space create a near-spherical isotropic noise field. Superdirectional beamforming utilizes an isotropic scattering noise field matrix to suppress the influence of noise from all directions on the target signal, outputting an array gain signal with higher "directivity" than conventional beamforming, thereby improving noise reduction performance.

[0119] In this embodiment, a spherical noise field matrix is ​​generated based on a second steering vector that incorporates the directional characteristics of a directional microphone. The spherical noise field matrix can be defined as the integral of the fundamental product between the second steering vector and its conjugate matrix. Specifically, the spherical noise field matrix can be expressed using the following formula:

[0120]

[0121] Where θ represents the variable of signal incident direction and the variable of signal incident elevation angle.

[0122] Step S1075: Determine the weight vector based on the spherical noise field matrix and the second steering vector.

[0123] In this embodiment, the weight vector can be determined as follows: obtain the first fundamental product between the inverse matrix of the spherical noise field matrix and the second guiding vector; obtain the second fundamental product between the conjugate matrix of the second guiding vector, the inverse matrix of the spherical noise field matrix, and the second guiding vector; and use the ratio of the first fundamental product to the second fundamental product as the weight vector. Specifically, the weight vector can be calculated using the following formula:

[0124]

[0125] Where Г represents the spherical noise field matrix, and ω represents the angular frequency; Indicates the second steering vector. θ d Indicates the direction of the target. This indicates the elevation angle of the target signal incident.

[0126] The derivation of the above formula can be achieved using the following formula:

[0127]

[0128] Constraints: By solving the objective function (optimization problem) of the super-directive beamformer, the maximization of the directivity factor (DF) can be obtained. Here, B represents the beam pattern in spherical coordinates, which can be expressed by the following formula:

[0129] In the above formula, The directivity factor DF is defined as the ratio between the signal power of the microphone array output in the desired directional direction and the average power in all directions, and can be expressed by the following formula:

[0130]

[0131] The above optimization problem can be expressed as the following formula:

[0132] Minh(ω) H Г(ω)h(ω)

[0133] Constraints:

[0134] Therefore, the super-directive beamformer of the microphone array is represented as:

[0135]

[0136] As can be seen from the above, the beamforming method provided in this application extends the existing super-directional beamforming method by introducing a degree of freedom, which is the directional microphone response matrix. In fact, when the microphone response matrix is ​​attenuated to a matrix (p=1 omnidirectional microphone), the existing super-directional beamforming method is a special case of the beamforming method provided in the embodiments of this application.

[0137] The differential beamforming method provided in this application can be evaluated using white noise gain (WNG), beam pattern, and directivity factor (DF). White noise refers to noise whose power spectral density is constant throughout the entire frequency domain; random noise with the same energy density at all frequencies is called white noise. WNG can evaluate the beamformer's ability to suppress spatially independent noise and its sensitivity to certain defects, such as noise generated by the directional microphone itself and microphone position errors. The beamformer's ability to suppress spatially independent noise is related to the beamforming weight vector. WNG can be defined using the following formula:

[0138]

[0139] The beam pattern describes the directional sensitivity of a beamformer to a plane wave incident on the array from an incident angle θ. The beam model can be defined using the following formula:

[0140]

[0141] Where B represents the beam pattern in spherical coordinates, and θ represents the azimuth angle. Indicates the elevation angle.

[0142] DF is the ratio between the signal power output by the array at the desired steering vector and the average power in all directions, and can be defined by the following formula:

[0143]

[0144] DL[h(ω)]=10log10(DF[h(ω)])

[0145] Experiments have shown that the method provided in this application can significantly improve the white noise gain at low frequencies and make the beam pattern more frequency invariant while achieving the same DF index as the prior art, thereby improving the robustness of super-directional beamforming.

[0146] As can be seen from the above embodiments, the beamforming method provided in this application obtains the incident direction and elevation angle of the target signal, microphone pointing direction information, and microphone array shape information; determines the response vector of the microphone array based on the microphone pointing direction information, the incident direction and elevation angle of the target signal; determines the first steering vector of the microphone array based on the microphone array shape information, the incident direction and elevation angle of the target signal; and determines the weight vector of the super-directional beamformer based on the response vector and the first steering vector. This processing method enables super-directional beamforming based on a directional microphone array, and adds a degree of freedom to the super-directional beamforming process, namely the amplitude response obtained based on the microphone pointing information. This results in higher white noise gain in the low-frequency band and a higher directivity factor in the high-frequency band, with a more frequency-invariant beam pattern; therefore, it can effectively improve the robustness of super-directional beamforming.

[0147] Second Embodiment

[0148] In the above embodiments, a beamforming method is provided. Correspondingly, this application also provides a beamforming apparatus. This apparatus corresponds to the embodiments of the method described above. Since the apparatus embodiments are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments. The apparatus embodiments described below are merely illustrative.

[0149] Please refer to Figure 5 This is a schematic diagram of an embodiment of the beamforming apparatus of this application. In this embodiment, the apparatus includes: an information acquisition unit 501, a response vector determination unit 503, a first guide vector determination unit 505, and a weight vector determination unit 507.

[0150] The system includes a parameter acquisition unit for acquiring the target signal incident direction and target signal incident elevation angle, microphone pointing direction information, and microphone array shape information; a response vector determination unit for determining the response vector of the microphone array based on the microphone pointing direction information, target signal incident direction, and target signal incident elevation angle; a first steering vector determination unit for determining the first steering vector of the microphone array based on the microphone array shape information, target signal incident direction, and target signal incident elevation angle; and a weight vector determination unit for determining the weight vector of the super-pointing beamformer based on the response vector and the first steering vector.

[0151] In one example, the directional microphone array includes a ring array; the microphone pointing direction information is obtained by obtaining a first angle between the microphone and the horizontal direction and a second angle between the microphone's orientation and the diameter direction; the sum of the first angle and the second angle is used as the microphone pointing direction information.

[0152] In one example, the response vector determination unit is specifically used to obtain the angle between the incident direction of the target signal and the pointing direction of the microphone; and to obtain the amplitude response of the directional microphone based on the angle and the incident elevation angle of the target signal.

[0153] In one example, the directional microphone is a first-order directional microphone; the response vector determination unit is specifically used to determine the response vector of the microphone array based on the microphone pointing direction information, the incident direction of the target signal, the incident elevation angle of the target signal, and the coefficients of the first-order directional microphone.

[0154] In one example, the device further includes a parameter setting unit for setting different coefficients for first-order directional microphones for different directional microphones.

[0155] In one example, the directional microphone array is a uniformly arranged ring array; the first guide vector determinant is specifically used to obtain the third angle between the incident direction of the target signal and the first angle between the microphone and the first angle, where the first angle refers to the angle between the microphone and the horizontal direction of the ring array; the first guide vector of the directional microphone is obtained based on the radius information of the ring array, the third angle and the incident elevation angle of the target signal.

[0156] In one example, the directional microphone array is a linear array; the first guide vector determination unit is specifically used to obtain the distance value between each microphone and a reference microphone based on the shape information of the microphone array, wherein the reference microphone may be the microphone at the first position of the linear array from left to right; and to obtain the first guide vector based on the distance value, the incident direction of the target signal and the incident elevation angle of the target signal.

[0157] In one example, the weight vector determination unit is used to determine a second steering vector based on the response vector and the first steering vector; determine a spherical noise field matrix based on the second steering vector; and determine the weight vector based on the spherical noise field matrix and the second steering vector.

[0158] Third Embodiment

[0159] In the above embodiments, a beamforming method is provided. Correspondingly, this application also provides a sound pickup device. This device corresponds to the embodiments of the above method. Since the device embodiments are basically similar to the method embodiments, the description is relatively simple, and relevant details can be found in the description of the method embodiments. The device embodiments described below are merely illustrative.

[0160] This application also provides a sound pickup device, including: a loudspeaker; a directional microphone array; a processor; and a memory. The memory stores a program for implementing the beamforming method described above, and the terminal, when powered on, runs the program of the method through the processor.

[0161] The electronic device may be an audio / video conferencing terminal or a sound pickup device.

[0162] Although this application discloses preferred embodiments as described above, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the scope of protection of this application should be determined by the scope defined in the claims of this application.

[0163] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0164] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0165] 1. Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include non-transitory computer-readable media, such as modulated data signals and carrier waves.

[0166] 2. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A beamforming method for a sound pickup device, the sound pickup device comprising a directional microphone array, characterized in that, include: Acquire the target signal incident direction and target signal incident elevation angle, microphone pointing direction information, and microphone array shape information; Obtain the angle between the microphone pointing direction and the incident direction of the target signal; Based on the included angle and the incident elevation angle of the target signal, the amplitude response of the directional microphone is obtained, and the amplitude responses of multiple directional microphones constitute the response vector of the microphone array. Based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal, determine the first steering vector of the microphone array; The weight vector of the super-directing beamformer is determined based on the response vector and the first steering vector.

2. The method according to claim 1, characterized in that, The directional microphone array includes a circular array; The microphone pointing direction information is obtained in the following way: Obtain the first angle between the microphone and the horizontal direction and the second angle between the microphone orientation and the diameter direction; The sum of the first included angle and the second included angle is used as the microphone pointing direction information.

3. The method according to claim 1, characterized in that, The directional microphone is a first-order directional microphone; The step of obtaining the amplitude response of the directional microphone based on the included angle and the incident elevation angle of the target signal includes: The amplitude response of the directional microphone is obtained based on the included angle, the incident elevation angle of the target signal, and the coefficient of the first-order directional microphone.

4. The method according to claim 3, characterized in that, Also includes: Set different coefficients for first-order directional microphones for different directional microphones.

5. The method according to claim 1, characterized in that, The directional microphone array is a circular array; The step of determining the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal includes: Obtain the third angle between the incident direction of the target signal and the first angle between the microphone and the microphone, where the first angle is the angle between the microphone and the horizontal direction of the circular array. Based on the radius information of the circular array, the third included angle, and the incident elevation angle of the target signal, the first guide vector of the directional microphone is obtained.

6. The method according to claim 1, characterized in that, The directional microphone array is a linear array; The step of determining the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal, and the incident elevation angle of the target signal includes: Based on the microphone array shape information, the distance values ​​between each microphone and the reference microphone are obtained, wherein the reference microphone is the microphone that is first in the linear array from left to right. The first guiding vector is obtained based on the distance value, the incident direction of the target signal, and the incident elevation angle of the target signal.

7. The method according to claim 1, characterized in that, The step of determining the weight vector of the super-directing beamformer based on the response vector and the first steering vector includes: The second steering vector is determined based on the response vector and the first steering vector; Determine the spherical noise field matrix based on the second guiding vector; The weight vector is determined based on the spherical noise field matrix and the second steering vector.

8. A beamforming apparatus, characterized in that, For use in a sound pickup device, the sound pickup device comprising a directional microphone array, characterized in that it includes: The parameter acquisition unit is used to acquire the incident direction and elevation angle of the target signal, microphone pointing direction information, and microphone array shape information; The response vector determination unit is used to obtain the angle between the microphone pointing direction information and the incident direction of the target signal; based on the angle and the incident elevation angle of the target signal, the amplitude response of the directional microphone is obtained, and the amplitude responses of multiple directional microphones constitute the response vector of the microphone array. The first steering vector determination unit is used to determine the first steering vector of the microphone array based on the microphone array shape information, the incident direction of the target signal and the incident elevation angle of the target signal; The weight vector determination unit is used to determine the weight vector of the super-directing beamformer based on the response vector and the first steering vector.

9. A sound pickup device, characterized in that, include: speaker; Directional microphone array; processor; as well as A memory for storing a program for implementing the method according to any one of claims 1-7, wherein the pickup device is powered on and the program for running the method is executed by the processor.

Images