Robust beam forming method and device for microphone array, equipment and medium

Abstract

The invention provides a robust beam forming method and device for a microphone array, equipment and a medium, and the method comprises the steps: setting the microphone array, and obtaining the audio information of the microphone array; converting a time domain signal of the microphone array into a frequency domain signal through a short-time Fourier transform technology; determining an optimization problem by using the microphone array, and solving a robust super-directivity beam former for the optimization problem; and performing spatial filtering on the frequency domain signal of the microphone array through the robust super-directivity beam former to obtain a frequency domain output signal, and inversely transforming the frequency domain output signal into a time domain output signal. According to the beam forming method and the beam forming device, the problem that the applicable array structure range of the beam forming method in the prior art is limited is solved, and the capabilities of flexibly controlling the beam steering direction and freely controlling the robustness are realized.

Description

Technical Field

[0001] The present invention relates to the field of speech signal processing technology, and in particular to a robust beamforming method, device, equipment and medium for a microphone array. Background Art

[0002] Microphone array beamforming technology is widely used in sensor array systems to improve the quality of target speech by suppressing interference and noise. Many algorithms have been proposed in this field, including delay-sum beamforming, super-directional beamforming, and differential beamforming. Among them, differential beamforming is particularly attractive because of its ability to provide high directional gain and wideband frequency-invariant beam patterns, making it very suitable for the application of small-sized microphone arrays to achieve high-fidelity speech signal acquisition. For example, in applications such as voice assistants, remote conferencing, and smart speakers, microphone array technology uses multiple microphones to achieve long-distance sound pickup and reduce background noise and echo, providing clearer and more accurate speech signals.

[0003] However, differential microphone arrays currently face two major challenges: first, achieving arbitrary angle pointing in a plane; second, reducing or avoiding low-frequency white noise amplification. For challenge 1, current research has shown that annular or concentric annular microphone arrays can achieve arbitrary angle pointing in a plane, but in practical applications, they usually face irregular array structures in the plane. Therefore, the design of a robust differential beamformer for microphone arrays with arbitrary geometric structures in a plane needs to be solved urgently.

[0004] There is currently no effective solution to the problem that existing beamforming methods have limited applicability. Summary of the invention

[0005] The present invention provides a robust beamforming method, device, equipment and medium for a microphone array, which are used to solve the defect of limited application scope of the beamforming method in the prior art, and realize the ability to flexibly control the beam steering direction and freely control the robustness.

[0006] Compared with a single microphone, the advantage of a microphone array is its multi-channel configuration, which provides richer information and higher flexibility for implementing various acoustic signal processing tasks. The sound signals received by microphones at different positions include redundancy, which helps to improve the robustness of the signal. By processing the multiple received signals, noise can be suppressed more effectively, and it can also be used to calculate the estimate of the sound source to achieve certain goals, such as speech noise reduction / enhancement, dereverberation, sound source localization and tracking. And the microphone array can be communicatively coupled to a processing device (such as a digital signal processor (DSP) or a central processing unit CPU), which contains circuits that can be programmed to implement a beamformer to calculate the source estimate.

[0007] A beamformer is a spatial filter that forms a beam in a specific direction mainly by weighting and phase-adjusting the sound signal received by a multi-channel microphone array. This can selectively enhance or suppress the sound signal in space, allowing the system to more effectively capture the sound source in a specific direction. However, the existing array structures are generally regular arrays such as linear arrays and circular arrays, and the corresponding beamforming methods cannot be simply extended to microphone arrays of arbitrary geometric structures. When faced with an irregularly distributed microphone array, the performance will be severely degraded, and the beam steering capability will also be limited. In order to solve this practical problem, the present invention develops a differential beamforming method based on a microphone array of arbitrary planar geometric structure, and performs a robust design on it.

[0008] In a first aspect, the present invention provides a robust beamforming method for a microphone array, comprising: Setting a microphone array and obtaining audio information of the microphone array; Converting the time domain signal of the microphone array into a frequency domain signal by short-time Fourier transform technology; Using the microphone array, determining an optimization problem, and solving a robust super-directional beamformer for the optimization problem; The frequency domain signal of the microphone array is spatially filtered by the robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal.

[0009] According to a robust beamforming method for a microphone array provided by the present invention, a time domain signal of the microphone array is converted into a frequency domain signal by using a short-time Fourier transform technology, comprising: Given a multi-channel input signal, the multi-channel input signal is converted from a time domain signal to a frequency domain signal using a short-time Fourier transform technique, and the magnitude of the frequency domain signal is determined.

[0010] According to a robust beamforming method of a microphone array provided by the present invention, the microphone array is used to determine an optimization problem, including: For the microphone array, from the perspective of minimum beam mean square error, constraining the white noise gain to meet a minimum threshold, and expecting the directional signal to be distortion-free, determining an initial optimization problem, and designing an initial beamformer for the initial optimization problem; The initial beamformer is decomposed into a delayed-sum beamformer and a reduced-rank beamformer, and the result is introduced into the initial optimization problem to generate a transformed optimization problem.

[0011] According to a robust beamforming method for a microphone array provided by the present invention, the initial optimization problem is: while minimizing the error between an ideal beam pattern and an actual beam pattern, constraining the white noise gain to be within an ideal range and enabling distortion-free transmission of signals in a desired direction.

[0012] According to a robust beamforming method for a microphone array provided by the present invention, a robust super-directional beamformer is generated for the optimization problem, comprising: Constructing a cost function based on the optimization problem by Lagrange multiplier method and converting the optimization problem into a standard form; Based on the standard form of the optimization problem, a closed-form solution of the robust super-directional beamformer is obtained.

[0013] According to a robust beamforming method for a microphone array provided by the present invention, frequency domain signals of the microphone array are spatially filtered by the robust super-directional beamformer to obtain frequency domain output signals, comprising: Traversing each frame of the frequency domain signal of the microphone array, traversing each frequency point for each frame of the frequency domain signal, and intercepting the target signal; The target signal is weighted filtered using the robust super-directional beamformer to obtain a corresponding frequency domain output signal until the weighted filtering operation is completed for each frequency point of each frame.

[0014] According to a robust beamforming method for a microphone array provided by the present invention, the frequency domain output signal is inversely transformed into a time domain output signal, comprising: The frequency domain output signal is converted into a time domain output signal by inverse short-time Fourier transform technology.

[0015] In a second aspect, the present invention further provides a robust beamforming device for a microphone array, characterized in that it comprises: An acquisition module, used to set a microphone array and acquire audio information of the microphone array; A conversion module, used to convert the time domain signal of the microphone array into a frequency domain signal by short-time Fourier transform technology; An optimization module, configured to determine an optimization problem using the microphone array, and solve a robust super-directional beamformer for the optimization problem; The processing module is used to perform spatial domain filtering on the frequency domain signal of the microphone array through the robust super-directional beamformer to obtain a frequency domain output signal, and inversely transform the frequency domain output signal into a time domain output signal.

[0016] In a third aspect, the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the robust beamforming method for the microphone array as described in the first aspect above is implemented.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the robust beamforming method for a microphone array as described in the first aspect above.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the robust beamforming method for a microphone array as described in the first aspect above.

[0019] Compared with the prior art, the present invention has the following beneficial effects: The robust beamforming method for a microphone array provided by the present invention approximates an ideal beam pattern by utilizing Jacobi-Anger expansion to achieve consistent performance for microphone arrays of arbitrary structures, while constraining the white noise gain within a reasonable range, thereby constructing an optimization problem, and then converting the optimization problem into a quadratic eigenvalue problem to construct a robust super-directional beamformer, and performing spatial domain filtering through the robust super-directional beamformer, thereby solving the problem of limited applicability of existing beamforming methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 is a flow chart of a robust beamforming method for a microphone array provided by the present invention; Figure 2 is a schematic diagram of a microphone array of a planar arbitrary geometric structure in an embodiment of the present invention; Figure 3 It is a schematic diagram of a curve showing a change of a directivity factor DF (directivity factor) and a white noise gain WNG (white noise gain) of a PDMA (planar differential microphone array) as a function of frequency in an embodiment of the present invention; Figure 4 is a curve diagram showing changes of the directivity factor DF and the white noise gain WNG as a function of frequency under different expansion orders of the PDMA in an embodiment of the present invention; Figure 5 is the beam pattern when the PDMA points to -30° in an embodiment of the present invention; Figure 6 is the beam pattern when the PDMA points to -120° in an embodiment of the present invention; Figure 7 is the beam pattern when the PDMA is pointed at 120° in an embodiment of the present invention; Figure 8 is the beam pattern when the PDMA is pointed at 30° in an embodiment of the present invention; Fig. 9 is a structural block diagram of a robust beamforming device of a microphone array provided by the present invention; Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The performance of a microphone array can be reflected in its beam pattern. Figure 1 This aspect can be quantified by the directivity factor (DF), which is the ability of the beam pattern to maximize the ratio of its sensitivity in the observation direction to its average sensitivity in all directions. The observation direction is the angle of incidence of the sound signal with maximum sensitivity. However, since the microphone array is very sensitive to the noise generated by the hardware components of each microphone itself. This effect is called white noise gain (WNG). The design of microphone array beamformers can focus on finding the optimal beamforming filter with a specified array geometry under certain conditions. However, this is usually based on the traditional method of designing regular microphone arrays. For irregular planar arrays of arbitrary geometry, the design of the beamformer is first achieved by expanding the Jacobi-Anger series and then approximating the actual beam to the ideal beam pattern. However, the beamformer designed in this way faces the problem of poor robustness, which is manifested in the severe amplification of white noise, especially in the low frequency band.

[0024] Therefore, the present invention provides a robust beamforming method for a microphone array based on a planar differential microphone array (PDMA). Figure 1 is a flow chart of a robust beamforming method for a microphone array provided by the present invention, such as Figure 1 As shown, the method comprises the following steps: Step S101, setting a microphone array and obtaining audio information of the microphone array.

[0025] Step S102: Convert the time domain signal of the microphone array into a frequency domain signal by using short-time Fourier transform technology.

[0026] Step S103: using the microphone array, determining an optimization problem, and solving a robust super-directional beamformer for the optimization problem.

[0027] Step S104: spatially filter the frequency domain signal of the microphone array by a robust super-directional beamformer to obtain a frequency domain output signal, and inversely transform the frequency domain output signal into a time domain output signal.

[0028] In this method, first, in an actual acoustic scene (such as a classroom, conference room, etc.), the designed differential microphone array with arbitrary geometric structure is placed on the table to record audio information. Then, the time domain signal of the microphone array is converted into a frequency domain signal through the short-time Fourier transform (STFT) technology, and the number of channels is recorded as , the frequency point number is recorded as K , the frame length is recorded as N , then the size of the frequency domain signal is . Then, for the microphone array designed above and the final optimization target, the optimization problem is determined, and the corresponding robust super-directional beamformer is constructed. For example, in order to reduce the influence of white noise in the beamforming process, the optimization problem is to keep the white noise gain within a reasonable range. Correspondingly, the constructed robust super-directional beamformer can control the influence of the white noise gain. Finally, the frequency domain signal of the microphone array is spatially filtered by the robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal. In the above process, the Jacobi-Anger expansion is used to approximate the ideal beam pattern to achieve consistent performance for microphone arrays of arbitrary structures, while constraining the white noise gain within a reasonable range, so as to construct an optimization problem, and then this optimization problem is converted into a quadratic eigenvalue problem to construct a robust super-directional beamformer, and the robust super-directional beamformer is used to perform spatial filtering, which solves the problem of limited applicability of existing beamforming methods.

[0029] In some of the embodiments, the time domain signal of the microphone array is converted into a frequency domain signal by short-time Fourier transform technology, including: given a multi-channel input signal, using short-time Fourier transform technology to convert the multi-channel input signal from a time domain signal to a frequency domain signal, and determining the size of the frequency domain signal.

[0030] For example, Figure 2 is a schematic diagram of a microphone array of a planar arbitrary geometric structure in an embodiment of the present invention, such as Figure 2 For ease of description and without limitation, On the plane, the origin is used as the reference point to measure the position of the subsequent microphones. The distance from the microphone sensor to the reference position , No. The angle between the microphones and the reference point The direction of the source signal to the array can be described by the azimuth The steering vector represents the relative phase shift of the incident far-field waveform in the array. As mentioned above, the steering vector of the array can be defined as:

[0031] in, is an imaginary unit, , is the angular frequency, is the temporal frequency, It is The distance from the microphone to the origin, It is The azimuth of the microphone. is the speed of sound in air, usually assumed to be 340 m / s.

[0032] The developed array has flexible steering characteristics in space, so it can be assumed that the sound source is transmitted to the array from any direction. Therefore, the observation signal vector of the array can be expressed as: , in, Indicates m The received signal of the microphone is represents the propagation vector of the signal, is the related source signal, is the noise signal vector, which is defined as Similarly, the output of the observation signal after the beamformer can be expressed as:

[0033] in is an estimate of the expected signal, H represents the conjugate transpose of a matrix or vector, , Is the length The beamformer of M The spatial filter coefficients for each microphone.

[0034] The post-processor can convert the estimated value (for each of the multiple frequency bands) Convert to the time domain to provide a representation of Estimated sound source.

[0035] This PDMA can be associated with a beam pattern that reflects the beamformer's ability to transmit signals from a specific angular direction ( ) is the sensitivity of a plane wave incident on the array. In the above PDMA, from the angle The beam pattern of an incident plane wave can be defined as:

[0036] in, represents the beam pattern, which reflects the filter's ability to enhance or suppress signals from different directions in space. DF represents the ability of the beamformer to suppress spatial noise from directions other than the observation direction. For example, the DF associated with PDMA mentioned above can be written as:

[0037] in, is the pseudo-coherence matrix of the noise signal in the diffuse noise field (size is ). The i , j ) elements can be represented as: ,and It is i The microphone sensor and j The distance between the microphone sensors.

[0038] WNG is used to evaluate the sensitivity of the beamformer to certain inherent imperfections of PDMA (e.g., noise from its own hardware). As mentioned above, WNG associated with PDMA can be written as:

[0039] To simplify the expression, we omit the angular frequency in the following text. .

[0040] In some of the embodiments, a microphone array is used to determine an optimization problem, including: for the microphone array, from the perspective of minimum beam mean square error, constraining the white noise gain to meet a minimum threshold, and expecting the directional signal to be distortion-free, determining an initial optimization problem, and designing an initial beamformer for the initial optimization problem; decomposing the initial beamformer into the form of the sum of a delayed sum beamformer and a reduced rank beamformer, and bringing it into the initial optimization problem to generate a transformed optimization problem.

[0041] Specifically, the initial optimization problem is: while minimizing the error between the ideal beam pattern and the actual beam pattern, constrain the white noise gain to be within the ideal range and enable the signal to be transmitted without distortion in the desired direction.

[0042] The specific design is as follows: first, based on the ideal beam pattern, the Jacobi-Anger series expansion is performed on the steering vector, and then the beam pattern of the beamformer is made close to the ideal beam pattern, while constraining the white noise gain to be no less than the minimum threshold. , and ensure that the signal in the desired direction is transmitted without distortion, and construct an optimization problem. Based on the ideal beam pattern, we can get:

[0043] in:

[0044]

[0045]

[0046] The actual beam pattern can be obtained by expanding the Jacobi-Anger series:

[0047] in:

[0048] .

[0049] From the perspective of minimum mean square beam error, the error between the expected beam and the actual beam is minimized, and the robustness constraint is , and the desired direction distortion-free constraint is added to the optimization process, the following optimization problem can be obtained:

[0050] For example, to solve this initial optimization problem, we will It is represented by the sum of a delay-sum beamformer and a reduced-rank beamformer, that is: , in, is the delay-sum beamformer, The matrix size is yes The null space of Is a long The filter, and .

[0051] Then substitute the above formula into the optimization problem, and the initial optimization problem can be rewritten as:

[0052] in:

[0053] On this basis, a robust super-directional beamformer is generated for the optimization problem, including: constructing a cost function based on the optimization problem through the Lagrange multiplier method, and converting the optimization problem into a standard form; based on the standard form of the optimization problem, a closed-form solution of the robust super-directional beamformer is obtained.

[0054] Exemplarily, the cost function is constructed using the Lagrange multiplier method:

[0055] in, is the Lagrange multiplier. Then the cost function can be differentiated, and the optimization problem is transformed into a standard quadratic eigenvalue problem (QEP), which can be solved to obtain a closed-form solution for a robust beamformer for arbitrary PDMA.

[0056] Figure 3 1 is a schematic diagram of a curve showing a change in the directivity factor DF and the white noise gain WNG of the PDMA as a function of frequency in an embodiment of the present invention. It can be seen from the curve that the beamformer implements the constraint on the WNG value. Figure 4 1 is a curve diagram showing the change of the directivity factor DF and the white noise gain WNG as a function of frequency under different expansion orders of the PDMA in the embodiment of the present invention. It can be seen from the curve diagram that all beamformers meet the preset WNG constraints under different expansion orders. Figure 5-8 1 is the beam pattern of the PDMA at different pointing angles in the embodiment of the present invention. It can be seen from the figure that the beam former can achieve consistency of the beam pattern when pointing at any angle.

[0057] In some embodiments, a frequency domain output signal is obtained by spatially filtering the frequency domain signal of the microphone array through a robust super-directional beamformer, including: traversing each frame of the frequency domain signal of the microphone array, traversing each frequency point for each frame of the frequency domain signal, and intercepting the target signal; using a robust super-directional beamformer to perform weighted filtering on the target signal and obtain a corresponding frequency domain output signal, until the weighted filtering operation is completed for each frequency point of each frame. On this basis, the frequency domain output signal is inversely transformed into a time domain output signal, including: converting the frequency domain output signal into a time domain output signal through an inverse short-time Fourier transform technique.

[0058] For example, from arrive Traverse each frame of the frequency domain signal, and when executing to the current frame, arrive Traverse each frequency point of the conversion domain signal in the current frame and cut off the size The signal is weighted filtered using the obtained robust super-directional beamformer weight vector, and the filtered frequency domain output signal is stored; the above process is repeated until the conversion frequency point is completed. Then end this round of traversal; repeat the above process until the number of frames is completed Finally, the inverse STFT technique is used to inversely transform the frequency domain output signal into a time domain output signal.

[0059] In summary, the present invention proposes a robust beamforming method based on a differential microphone array with arbitrary planar geometry, which establishes an optimization problem by minimizing the error between the ideal beam pattern and the actual beam pattern while constraining the white noise gain to be at an ideal level and transmitted without distortion in the desired direction. By constructing a standard form QEP and solving the optimization problem, a robust differential beamformer is obtained.

[0060] The present invention also provides a robust beamforming device for a microphone array. The robust beamforming device for the microphone array provided by the present invention is described below. The robust beamforming device for the microphone array described below and the robust beamforming method for the microphone array described above can be referred to each other. Fig. 9 is a structural block diagram of a robust beamforming device of a microphone array provided by the present invention, such as Fig. 9 As shown, the device comprises: An acquisition module 901 is used to set a microphone array and acquire audio information of the microphone array; A conversion module 902, configured to convert the time domain signal of the microphone array into a frequency domain signal by using a short-time Fourier transform technique; An optimization module 903 is used to determine an optimization problem using a microphone array, and solve a robust super-directional beamformer for the optimization problem; The processing module 904 is used to perform spatial filtering on the frequency domain signal of the microphone array through a robust super-directional beamformer to obtain a frequency domain output signal, and inversely transform the frequency domain output signal into a time domain output signal.

[0061] When the device is used, first, the acquisition module 901 places the designed differential microphone array of arbitrary geometric structure on the desktop in an actual acoustic scene (such as a classroom, conference room, etc.) to record audio information. Then, the conversion module 902 converts the time domain signal of the microphone array into a frequency domain signal through the short-time Fourier transform technology (STFT). The multi-channel number is recorded as , the frequency point number is recorded as K , the frame length is recorded as N , then the size of the frequency domain signal is . Then, the optimization module 903 determines the optimization problem for the microphone array designed above and the final optimization target, and constructs the corresponding robust super-directional beamformer. For example, in order to reduce the influence of white noise amplification during the beamforming process, the optimization problem is to keep the white noise gain within a reasonable range, and correspondingly, the constructed robust super-directional beamformer can control the value of the white noise gain. Finally, the processing module 904 performs spatial filtering on the frequency domain signal of the microphone array through the robust super-directional beamformer to obtain a frequency domain output signal, and inversely transforms the frequency domain output signal into a time domain output signal. In the above process, the Jacobi-Anger expansion is used to approximate the ideal beam pattern to achieve consistent performance for microphone arrays of arbitrary structures, while constraining the white noise gain within a reasonable range, so as to construct an optimization problem, and then the optimization problem is converted into a quadratic eigenvalue problem to construct a robust super-directional beamformer, and the robust super-directional beamformer is used to perform spatial filtering, thereby solving the problem of limited application scope of the existing beamforming method.

[0062] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10As shown, the electronic device may include: a processor (processor) 1001, a communication interface (Communications Interface) 1002, a memory (memory) 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004. The processor 1001 may call the logic instructions in the memory 1003 to execute the robust beamforming method of the microphone array, and the method includes: Set up a microphone array and obtain audio information from the microphone array; The time domain signal of the microphone array is converted into a frequency domain signal by short-time Fourier transform technology; Using microphone arrays, an optimization problem is identified and a robust super-directional beamformer is solved for the optimization problem. The frequency domain signal of the microphone array is spatially filtered by a robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal.

[0063] In addition, the logic instructions in the above-mentioned memory 1003 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0064] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the robust beamforming method of the microphone array provided by the above methods, and the method includes: Set up a microphone array and obtain audio information from the microphone array; The time domain signal of the microphone array is converted into a frequency domain signal by short-time Fourier transform technology; Using microphone arrays, an optimization problem is identified and a robust super-directional beamformer is solved for the optimization problem. The frequency domain signal of the microphone array is spatially filtered by a robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal.

[0065] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the robust beamforming method of the microphone array provided by the above methods, the method comprising: Set up a microphone array and obtain audio information from the microphone array; The time domain signal of the microphone array is converted into a frequency domain signal by short-time Fourier transform technology; Using microphone arrays, an optimization problem is identified and a robust super-directional beamformer is solved for the optimization problem. The frequency domain signal of the microphone array is spatially filtered by a robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal.

[0066] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0067] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robust beamforming method for a microphone array, characterized in that: include: Setting a microphone array and obtaining audio information of the microphone array; Converting the time domain signal of the microphone array into a frequency domain signal by short-time Fourier transform technology; Using the microphone array, determining an optimization problem, and solving a robust super-directional beamformer for the optimization problem; The frequency domain signal of the microphone array is spatially filtered by the robust super-directional beamformer to obtain a frequency domain output signal, and the frequency domain output signal is inversely transformed into a time domain output signal.

2. The robust beamforming method for a microphone array according to claim 1, characterized in that: The time domain signal of the microphone array is converted into a frequency domain signal by using a short-time Fourier transform technology, including: Given a multi-channel input signal, the multi-channel input signal is converted from a time domain signal to a frequency domain signal using a short-time Fourier transform technique, and the magnitude of the frequency domain signal is determined.

3. The robust beamforming method for a microphone array according to claim 1, characterized in that: Using the microphone array, an optimization problem is determined, including: For the microphone array, from the perspective of minimum beam mean square error, constraining the white noise gain to meet a minimum threshold, and expecting the directional signal to be distortion-free, determining an initial optimization problem, and designing an initial beamformer for the initial optimization problem; The initial beamformer is decomposed into a delayed-sum beamformer and a reduced-rank beamformer, and the result is introduced into the initial optimization problem to generate a transformed optimization problem.

4. The robust beamforming method for a microphone array according to claim 3, characterized in that: The initial optimization problem is: while minimizing the error between the ideal beam pattern and the actual beam pattern, constrain the white noise gain to be within the ideal range and enable the signal to be transmitted without distortion in the desired direction.

5. The robust beamforming method for a microphone array according to claim 1, characterized in that: Generating a robust super-directional beamformer for the optimization problem, including: Constructing a cost function based on the optimization problem by Lagrange multiplier method and converting the optimization problem into a standard form; Based on the standard form of the optimization problem, a closed-form solution of the robust super-directional beamformer is obtained.

6. The robust beamforming method for a microphone array according to claim 1, characterized in that: The method comprises: performing spatial filtering on the frequency domain signal of the microphone array by the robust super-directional beamformer to obtain a frequency domain output signal, comprising: Traversing each frame of the frequency domain signal of the microphone array, traversing each frequency point for each frame of the frequency domain signal, and intercepting the target signal; The target signal is weighted filtered using the robust super-directional beamformer to obtain a corresponding frequency domain output signal until the weighted filtering operation is completed for each frequency point of each frame.

7. The robust beamforming method for a microphone array according to claim 1, characterized in that: Inversely transforming the frequency domain output signal into a time domain output signal, comprising: The frequency domain output signal is converted into a time domain output signal by inverse short-time Fourier transform technology.

8. A robust beamforming device for a microphone array, characterized in that: include: An acquisition module, used to set a microphone array and acquire audio information of the microphone array; A conversion module, used to convert the time domain signal of the microphone array into a frequency domain signal by short-time Fourier transform technology; An optimization module, configured to determine an optimization problem using the microphone array, and solve a robust super-directional beamformer for the optimization problem; The processing module is used to perform spatial domain filtering on the frequency domain signal of the microphone array through the robust super-directional beamformer to obtain a frequency domain output signal, and inversely transform the frequency domain output signal into a time domain output signal.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the robust beamforming method for the microphone array according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robust beamforming method for the microphone array according to any one of claims 1 to 7 is implemented.

Images