Uniform area array constant beam width linear constraint minimum variance beam forming method and device

Abstract

The invention discloses a uniform area array constant beam width linear constraint minimum variance beam forming method and device, and the method comprises the steps: respectively and independently designing two stages of sub-arrays through employing the property that a planar array weight matrix can be represented as a two-dimensional convolution form of a two-stage sub-array weight matrix; the method has the advantages of high fixed beam robustness, low complexity and strong adaptive beam interference noise suppression capability, meets the requirements of constant beam width in an expected direction and formation of wide null suppression in an interference direction, is simple in design and low in complexity, and can support application with high real-time requirement. According to the invention, a two-dimensional Chebyshev windowing constant beam width fixed beam method is adopted, and window parameters adjusted along with frequency are utilized, so that not only is the beam width constant maintained, but also the sidelobe level can be effectively controlled.

Description

Technical Field

[0001] The present invention relates to the technical field of microphone array signal acquisition and processing, and in particular to a uniform array constant beam width linear constraint minimum variance beamforming method and device. Background Art

[0002] Existing broadband beamforming technologies are mainly divided into two categories: fixed beamforming technology and adaptive beamforming technology. Constant beamwidth fixed beamforming technology achieves frequency invariance of beamwidth over a wide frequency band. When the beam direction is not precisely aligned with the actual direction of the desired signal source, the constant beamwidth avoids signal distortion, but this type of technology is difficult to form a null in a specific direction to suppress interference signals. Another type of adaptive beamforming technology mainly uses linear constrained minimum variance (LCMV) beamforming technology, which can better form a null in the direction of the interference source to block directional interference, but cannot maintain a constant beamwidth over a wide frequency band. At the same time, the sidelobes in the non-interference direction are relatively high, especially when the number of array elements is large, resulting in a large covariance matrix with high inversion complexity and poor robustness, which is difficult to implement in practical applications. The latest beamforming technology imposes constraints on the desired direction and the interference direction, and then uses software tools such as cxv to solve the optimization problem to obtain the weight coefficients. This type of technology can better control the beam and null, but it still has problems such as high computational complexity and poor robustness. When the direction of the target sound source or the interference sound source changes in real time, the software tool needs to be re-called to re-solve the optimization problem. It is not flexible and is difficult to apply in situations with high real-time requirements. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention aims to provide a method and device for uniform array constant beamwidth linearly constrained minimum variance beamforming.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] A uniform array constant beamwidth linearly constrained minimum variance beamforming method comprises the following steps:

[0006] Step 1: A planar array with evenly distributed microphone elements collects sound signals in the environment, and converts each frame of sound signals collected by the planar array into a frequency domain through short-time Fourier transform processing;

[0007] Step 2: The planar array is regarded as a cascade representation of two-level planar sub-arrays, and the weight matrix of the planar array is represented as a two-dimensional convolution of the weight matrices of the two-level planar sub-arrays; the dimension parameters of the two-level planar sub-arrays are set respectively to obtain the corresponding sub-array steering matrix and sub-array steering vector;

[0008] Step 3, dividing the first-level planar subarray: select a first-level planar subarray on the planar array, obtain a number of overlapping first-level planar subarrays by translation, and use a two-dimensional Chebyshev windowed constant beamwidth beamforming method to perform weighted summation on the frequency domain signal of each first-level planar subarray to obtain the frequency domain output of each first-level planar subarray;

[0009] Step 4: The output signals of each first-stage planar subarray are combined into a second-stage planar subarray, a linear constrained minimum variance beamformer with null widening is constructed, and adaptive spatial filtering is performed to enhance the target direction signal and form wide null suppression in the interference direction;

[0010] Step 5: Perform inverse Fourier transform on the final beamforming output frequency domain signal calculated in step 4 to restore the time domain signal of each frame; at the same time, continuously determine whether the direction of the target sound source and the interference sound source pointed by the beam has changed. If not, there is no need to update the weight coefficients, and repeatedly execute steps 1 to 4 to output the speech enhancement signal; if there is a change, only steps 3 and 4 are needed to update the subarray beamforming weight coefficients, and then output the new speech enhancement signal.

[0011] Furthermore, the specific process of step 1 is:

[0012] The sound signal is collected by using a planar array with uniformly distributed microphone elements in the spatial topology. The microphone elements are uniformly distributed on the x-axis and y-axis two-dimensional planes, and are composed of M×N omnidirectional microphones. M is the number of rows of the planar array in the y-axis direction, N is the number of columns of the planar array in the x-axis direction, and the spacing between the microphone elements on the x-axis and y-axis is δ x and δ y ; Taking the microphone array element at the upper left corner of the plane array as the reference point and the origin of the coordinate system, each microphone array element can be represented by a serial number (m, n), m = 0, 1, ..., M-1, n = 0, 1, ..., N-1, where m and n represent the serial numbers of the row and column of the microphone array element in the plane array, respectively. The spatial position coordinates of the microphone array element (m, n) in the coordinate system can be expressed as (nδ x ,mδ y ,0), the time domain signal collected by the microphone array element (m,n) is transformed into the frequency domain form X by short-time Fourier transform mn (k), k is the frequency point f k The serial number, f k =kf s / N FFT , f s is the sampling rate, N FFT is the Fourier transform length; considering the far-field acoustic model, θ and φ are used to represent the elevation angle and azimuth angle of the incoming wave direction of the sound source, respectively, and the steering matrix of the M×N planar array is defined as:

[0013]

[0014] where D(f,θ,φ) is the array steering matrix, [D(f,θ,φ)] mn represents the element in the mth row and nth column of the array steering matrix, the wave number parameter c is the propagation speed of sound in the environment, and f is the frequency; the columns of the array steering matrix are stacked as a vector representation to obtain the corresponding array steering vector:

[0015]

[0016] Furthermore, the planar array beamforming is a spatial linear filtering of the array frequency domain signal. The spatial filtering weight coefficients of the planar array are represented by the weight matrix W:

[0017]

[0018] The weight matrix is ​​stacked column by column as a vector representation, resulting in a flat array of weight vectors:

[0019] w=[w 00 ,…,w m0 ,…,w (M-1)0 ,…,w mn ,…,w (M-1)(N-1) ] T (4)

[0020] The beam pattern corresponding to the weight vector of the planar array is B(θ,φ,w)=d H (θ, φ)w, the planar array is regarded as a cascade of two-stage planar sub-arrays, and its beam pattern is equivalent to the multiplication of two sub-beam patterns B(θ, φ, w) = B1(θ, φ, w1) B2(θ, φ, w2), and a fixed or adaptive sub-array beamformer is designed separately; the weight vectors of the two-stage planar sub-arrays are w1 and w2 respectively; the steering vectors of the two-stage planar sub-arrays are:

[0021]

[0022] Where M1 and N1 are the number of rows and columns of the first-level planar subarray, M2 and N2 are the number of rows and columns of the second-level planar subarray, and the beam pattern is expressed in the form of a Cauchy product:

[0023]

[0024] Among them, w mn is the weight coefficient of the microphone array element (m,n), and is the weight coefficient of the corresponding element of the two-level planar subarray. To simplify the expression, and According to formula (5), assuming that the number of rows and columns of the planar array and the two-level planar sub-array satisfy the relationship M = M1 + M2-1 and N = N1 + N2-1, the weight matrix of the planar array is expressed as the two-dimensional convolution of the weight matrix of the two-level planar sub-array:

[0025]

[0026] Among them, parameters a1 = max(0, m-1 + M2), a2 = min(m, M1-1), b1 = max(0, n-1 + N2) and b2 = min(n, N2-1).

[0027] Furthermore, the row and column parameters of the first-level planar subarray are M1=6, N1=8, the first-level planar subarray is a 6×8 planar array, and the corresponding row and column parameters of the second-level planar subarray can be calculated as M2=M+1-M1=3, N2=N+1-N1=3, then the second-level planar subarray is a 3×3 planar array.

[0028] Furthermore, the specific process of step 3 is:

[0029] 3.1) Divide the first-level plane subarray: Using the invariant property of the plane array formation, according to the row parameter M1 and column parameter N1 set in step 2, select consecutive N1 columns in consecutive M1 rows to form an M1×N1 subarray. By translation on the M×N plane array, (M-M1+1)(N-N1+1)=M2N2 first-level plane subarrays are obtained. Each first-level plane subarray is represented by a serial number (m',n')m'=0,1,...,M2-1,n'=0,1,...,N2-1, where n' and m' represent the serial numbers of the row and column of each first-level plane subarray, respectively. Then, the frequency domain representation of the (m,n)th microphone array element in the first-level plane subarray (m',n') is X (m '+m)( n '+n ) , where m = 0, 1, ..., M1-1, n = 0, 1, ..., N1-1;

[0030] 3.2) Calculate the Chebyshev coefficients of each first-level planar subarray in the x-axis direction, determine the x-axis main lobe width according to the design goal, and calculate the approximate intermediate value Get the Chebyshev weighted beam zero position:

[0031]

[0032] Construct the beam zero position steering vector matrix:

[0033]

[0034] Among them, the steering vector represented by each column in the matrix is:

[0035]

[0036] Then the Chebyshev coefficient in the x-axis direction is expressed as where e1 = [1, 0, ..., 0] T ;

[0037] 3.3) According to the design target y-axis main lobe width Calculate the Chebyshev coefficient h of each first-level plane subarray in the y-axis direction (y) :

[0038] Calculates approximate middle value Get the beam zero point position of each first-level planar subarray in the y-axis dimension:

[0039]

[0040] Construct the beam zero position steering vector matrix:

[0041]

[0042] Among them, the steering vector represented by each column in the matrix is:

[0043]

[0044] Then the Chebyshev coefficient of each first-level plane subarray in the y-axis direction can be expressed as

[0045] 3.4) The Chebyshev coefficient vector h of each first-level plane subarray is given by h (x) and h (y) Together, the elements in the vector are in and They are vector h (x) and h (y) The elements in; the weight vector w1 of the two-dimensional Chebyshev windowed beamformer is:

[0046] w1=diag{h}d1(θ s ,φ s )=Λd1(θ s ,φ s ) (10)

[0047] Among them, θ s ,φ sare the pitch angle and horizontal angle of the target sound source s, respectively, s ,φ s ) is calculated according to formula (5), Λ=diag{h} represents a diagonal matrix with vector h as the matrix diagonal element, and Chebyshev windowed beamforming is used to achieve effective control of side lobes while keeping the beam width unchanged in the frequency domain;

[0048] 3.5) The frequency domain signal of each first-level planar subarray is spatially filtered using the corresponding two-dimensional Chebyshev windowed beamforming weight vector, and the output signal of each first-level planar subarray (m', n') is obtained as follows:

[0049]

[0050] in, is the weight vector The corresponding element in (·) * denotes the conjugation of the elements in brackets.

[0051] Furthermore, the specific process of step 4 is:

[0052] 4.1) The output signal of the first-level planar sub-array n'=0,1,…,N2-1 form the second-level planar subarray, expressed as

[0053] 4.2) The incident direction of the interference sound source (θ j' ,φ j' ) to widen the possible incident area Θ of the signal of the interfering sound source j' , for Θ j' The interference steering vector d2(θ j' ,φ j' ), j'=1,2,…,J, where J represents the number of interference angles after discretization, d2(θ j' ,φ j' ) is calculated according to formula (6); the correlation matrix of the discrete interference steering vector is constructed as follows:

[0054]

[0055] 4.3) Performing eigenvalue decomposition on the correlation matrix C yields λ1≥λ2≥…≥λ J , where λ k is the eigenvalue of the decomposition, v k is the corresponding eigenvector. The eigenvector with the largest eigenvalue is selected as the basis vector to form the projection matrix. Let the parameter K traverse from 1 to J and determine whether it satisfies the following inequality:

[0056]

[0057] Among them, ε is a set constant. If the inequality is satisfied, the traversal process is stopped, and K is determined to be the number of selected large eigenvalues;

[0058] 4.4) Select the eigenvectors corresponding to the first K largest eigenvalues ​​to form the projection matrix Use the projection matrix to preprocess the second-level plane subarray to obtain the projected subarray signal The covariance matrix of the projected subarray signal is:

[0059]

[0060] Where R is the covariance matrix of the second-level plane subarray y1, and N is the number of snapshots for calculating the signal covariance matrix sampling;

[0061] 4.5) The covariance matrix of the projected subarray signal obtained in step 4.4) and the known second-level planar subarray target signal steering vector d2(θ s ,φ s ), the zero-stretched linear constrained minimum variance LCMV adaptive weight vector of the second-level planar subarray is calculated as:

[0062]

[0063] Among them, d2(θ s ,φ s ) is calculated according to formula (6), represents the inversion of the covariance matrix of the projected sub-array signal;

[0064] 4.6) Perform weighted summation on the second-level planar subarray to obtain the final beamforming output frequency domain signal

[0065] The present invention also provides a uniform planar array constant beamwidth linearly constrained minimum variance beamforming device for implementing the above method, comprising a planar array with uniformly distributed microphone array elements, a first-level planar subarray signal processing module, a second-level planar subarray signal processing module and a signal conversion processor.

[0066] The planar array is used to collect sound signals in the environment and convert each frame of sound signals into frequency domain signals;

[0067] The planar array is divided into a plurality of first-level planar sub-arrays, the first-level planar sub-array signal processing module is used to perform beamforming on each first-level planar sub-array, and the second-level planar sub-array signal processing module is used to perform beamforming on the frequency domain signal output by the first-level planar sub-array signal processing module to obtain a final beamforming output frequency domain signal;

[0068] The signal conversion processor is used to convert the beamforming output frequency domain signal into a time-frequency signal.

[0069] The beneficial effects of the present invention are:

[0070] 1. The present invention utilizes the property that the planar array weight matrix can be expressed as a two-dimensional convolution form of the two-level sub-array weight matrix, and designs the two-level sub-arrays separately, which has the advantages of high fixed beam robustness, low complexity and strong adaptive beam interference noise suppression capability, and at the same time meets the requirements of constant beam width in the desired direction and wide null suppression in the interference direction. The design is simple and low in complexity, and can support applications with high real-time requirements;

[0071] 2. The present invention adopts a two-dimensional Chebyshev windowed constant beam width fixed beam method, and uses the window parameters adjusted with the frequency to not only keep the beam width constant, but also effectively control the sidelobe level. Compared with the existing methods, it has the advantages of lower sidelobe level, higher directivity coefficient and white noise gain;

[0072] 3. The present invention can flexibly adjust the dimensional parameters of the adaptive beam subarray according to the number of interfering sound sources and the interference suppression requirements, make full use of its degrees of freedom to perform wide null setting and interference suppression, balance the method output noise reduction capability and implementation complexity, and is easy to implement with high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is an overall flow chart of the method of Example 1 of the present invention;

[0074] Figure 2 Schematic diagram of the planar array and sound source direction of the method of embodiment 1 of the present invention;

[0075] Figure 3 A schematic diagram of constructing the first-level planar subarray and the second-level planar subarray according to the method of embodiment 1 of the present invention;

[0076] Figure 4 Graphs showing broadband beam responses that vary with frequency and angle under the method of Embodiment 1 of the present invention and common beamforming methods;

[0077] Figure 5 is a comparison diagram of signal output power of the method of embodiment 1 of the present invention and a common beamforming method as the incident angle changes;

[0078] Figure 6 The figure is a comparison of output signal waveforms of three groups of beamformers in the method of embodiment 2 of the present invention;

[0079] Figure 7 Spectrograms of different methods in Example 2 of the present invention;

[0080] Figure 8 This is a schematic diagram of the device structure of Example 3 of the present invention. DETAILED DESCRIPTION

[0081] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0082] Example 1

[0083] This embodiment provides a uniform array constant beamwidth linear constrained minimum variance beamforming method, such as Figure 1 As shown, the following steps are included:

[0084] Step 1: A planar array with uniformly distributed microphone elements collects sound signals in the environment, and converts each frame of sound signals collected by the planar array into the frequency domain through short-time Fourier transform processing.

[0085] The spatial topology is a planar array with uniformly distributed microphone elements to collect sound signals. Figure 2 is a schematic diagram of the plane array and the sound source direction. The microphone array elements are evenly distributed on the two-dimensional plane of the x-axis and y-axis. It consists of M×N omnidirectional microphones. M is the number of rows of the plane array in the y-axis direction, N is the number of columns of the plane array in the x-axis direction, and the spacing between the microphone array elements on the x-axis and y-axis is δ x and δ y Taking the microphone array element in the upper left corner of the plane array as the reference point and the origin of the coordinate system, each microphone array element can be represented by a serial number (m, n), m = 0, 1, ..., M-1, n = 0, 1, ..., N-1, where m and n represent the serial numbers of the row and column of the microphone array element in the plane array, respectively. The spatial position coordinates of the microphone array element (m, n) in the coordinate system can be expressed as (nδ x ,mδ y ,0), the time domain signal collected by the microphone array element (m,n) is transformed into the frequency domain form X by short-time Fourier transform mn (k), k is the frequency point f k The serial number, f k =kf s / N FFT , f s is the sampling rate, N FFTis the Fourier transform length. Considering the far-field acoustic model, θ and φ represent the elevation angle and azimuth angle of the incoming wave direction of the sound source, respectively, the steering matrix of the M×N planar array is defined as:

[0086]

[0087] where j represents the imaginary unit, D(f,θ,φ) is the array steering matrix, [D(f,θ,φ)] mn represents the element in the mth row and nth column of the array steering matrix, the wave number parameter c is the propagation speed of sound in the environment, and f is the frequency. Stacking the columns of the array steering matrix into a vector representation gives the corresponding array steering vector:

[0088]

[0089] In this embodiment, M=8, N=10, δ x =0.04 m, δ y = 0.05 meters, there is a target sound source s in the space, the pitch angle and azimuth angle of the target sound source s wave direction are (θ s ,φ s )=(10°,180°), the elevation angle and azimuth angle of the wave directions of the two interfering sound sources j1 and j2 are and

[0090] Step 2: Consider the planar array as a cascade representation of two-level planar subarrays. The weight matrix of the planar array can be represented as a two-dimensional convolution of the weight matrices of the two-level planar subarrays. Set the dimension parameters of the two-level planar subarrays respectively to obtain the corresponding subarray steering matrix and subarray steering vector. The specific process is:

[0091] Planar array beamforming is a spatial linear filtering of the array frequency domain signal. The spatial filtering weight coefficients of the planar array can be expressed by the weight matrix W:

[0092]

[0093] The weight matrix is ​​stacked column by column as a vector representation, resulting in a flat array of weight vectors:

[0094] w=[w 00 ,...,w m0 ,...,w (M-1)0 ,...,w mn ,...,w (M-1)(N-1) ] T (4)

[0095] The beam pattern corresponding to the weight vector of the planar array is B(θ,φ,w)=d H (θ, φ)w, the planar array is regarded as a cascade of two-stage planar subarrays, and its beam pattern is equivalent to the multiplication of two sub-beam patterns B(θ, φ, w) = B1(θ, φ, w1) B2(θ, φ, w2). The two-stage planar subarrays are separately designed as fixed beamformers and adaptive beamformers, which respectively meet the requirements of constant beam width, controllable sidelobe suppression and directional interference nulling, thereby reducing the design complexity. The weight vectors of the two-stage planar subarrays are w1 and w2 respectively. Similar to formula (2), the steering vectors of the two-stage planar subarrays are:

[0096]

[0097] Where M1 and N1 are the number of rows and columns of the first-level planar subarray, M2 and N2 are the number of rows and columns of the second-level planar subarray, and the beam pattern is expressed in the form of a Cauchy product:

[0098]

[0099] Among them, w mn is the weight coefficient of the microphone array element (m,n), and They are the weight coefficients of the corresponding elements of the two-level planar subarrays. To simplify the expression, and Since the algorithm is independent and consistent at any frequency point, without loss of generality, the frequency point number k or frequency f is omitted in the formula of the method of this embodiment. According to formula (5), assuming that the number of rows and columns of the planar array and the two-level planar sub-array satisfies the relationship M=M1+M2-1 and N=N1+N2-1, the weight matrix of the planar array can be expressed as the two-dimensional convolution of the weight matrix of the two-level planar sub-array:

[0100]

[0101] Among them, parameters a1 = max(0, m-1 + M2), a2 = min(m, M1-1), b1 = max(0, n-1 + N2) and b2 = min(n, N2-1).

[0102] Therefore, the subarray beamformer can be designed individually according to the requirements of constant beam width and interference direction nulling in practical applications, which can reduce the computational complexity without affecting the overall performance.

[0103] The row and column number parameters of the two-level planar subarray are set. In this embodiment, the first-level planar subarray adopts a fixed beam method to form a main lobe with a constant beam width, and the side lobe needs to be reduced as much as possible. Therefore, the selected M1 and N1 values ​​are slightly larger, but must be smaller than M and N. The second-level planar subarray adopts an adaptive beamforming method. The number of microphone array elements of the second-level planar subarray should be reduced as much as possible while ensuring sufficient degrees of freedom. This can ensure that the null does not enter the main lobe area and achieve a balance between noise interference suppression and algorithm complexity.

[0104] In this embodiment, the row and column parameters of the first-level planar subarray are selected as M1=6, N1=8, and the first-level planar subarray is a 6×8 planar array. The corresponding row and column parameters of the second-level planar subarray can be calculated as M2=M+1-M1=3, N2=N+1-N1=3, and the second-level planar subarray is a 3×3 planar array.

[0105] Step 3, divide the first-level planar subarray: select a first-level planar subarray on the planar array, obtain several overlapping first-level planar subarrays by translation, and use the two-dimensional Chebyshev windowed constant beamwidth beamforming method to perform weighted summation on the frequency domain signal of each first-level planar subarray to obtain the frequency domain output of each first-level planar subarray. The specific process is:

[0106] 3.1) Divide the first-level planar subarray into Figure 3 As shown, by utilizing the translation-invariant property of the planar array formation, according to the row parameter M1 and column parameter N1 set in step 2, continuous N1 columns in continuous M1 rows are selected to form an M1×N1 subarray. By translation on the M×N planar array, (M-M1+1)(N-N1+1)=M2N2 first-level planar subarrays can be obtained. Each first-level planar subarray is represented by a serial number (m',n')m'=0,1,...,M2-1,n'=0,1,...,N2-1, where n' and m' represent the serial numbers of the row and column of each first-level planar subarray, respectively. Then, the frequency domain representation of the (m,n)th microphone array element in the first-level planar subarray (m',n') is X (m'+m)(n'+n) , where m=0,1,…,M1-1, n=0,1,…,N1-1.

[0107] 3.2) Calculate the Chebyshev coefficients of each first-level planar subarray in the x-axis direction. According to the design goal, the x-axis main lobe width is Calculates approximate middle value Get the Chebyshev weighted beam null position of each first-level planar subarray:

[0108]

[0109] Construct the beam zero position steering vector matrix:

[0110]

[0111] Among them, the steering vector represented by each column in the matrix is:

[0112]

[0113] Then the Chebyshev coefficient of each first-level plane subarray in the x-axis direction can be expressed as where e1 = [1, 0, ..., 0] T .

[0114] 3.3) Using the same method as in step 3.2), according to the y-axis main lobe width in the design target Calculate the Chebyshev coefficient h of each first-level plane subarray in the y-axis direction (y) :

[0115] Calculates approximate middle value Get the beam zero point position of each first-level planar subarray in the y-axis dimension:

[0116]

[0117] Construct the beam zero position steering vector matrix:

[0118]

[0119] Among them, the steering vector represented by each column in the matrix is:

[0120]

[0121] Then the Chebyshev coefficient of each first-level plane subarray in the y-axis direction can be expressed as

[0122] 3.4) The Chebyshev coefficient vector h of the first-level plane subarray can be expressed as h (x) and h (y) Together, the elements of vector h are n=0,1,…,N1-1,where and They are vector h (x) and h (y) The two-dimensional Chebyshev windowed beamformer weight vector w1 for each first-level plane subarray is:

[0123] w1=diag{h}d1(θ s ,φ s )=Λd1(θ s ,φs ) (10)

[0124] Among them, θ s =10° and φ s =180° is the pitch angle and horizontal angle of the target sound source s wave direction, d1(θ s ,φ s ) is calculated according to formula (5), Λ = diag{h} represents a diagonal matrix with vector h as the diagonal element of the matrix. The Chebyshev windowed beamforming can achieve effective control of the sidelobe while keeping the beam width unchanged in the frequency domain.

[0125] 3.5) The frequency domain signal of each first-level planar subarray is spatially filtered using the corresponding two-dimensional Chebyshev windowed beamforming weight vector, and the output signal of each first-level planar subarray (m', n') is obtained as follows:

[0126]

[0127] in, is the weight vector The corresponding element in (·) * denotes the conjugation of the elements in brackets.

[0128] Step 4: The output signals of each first-stage planar subarray are combined into a second-stage planar subarray, a linear constrained minimum variance beamformer with null widening is constructed, and adaptive spatial filtering is performed to enhance the target direction signal and form wide null suppression in the interference direction. The specific process is as follows:

[0129] 4.1) The output signal of the first-level planar sub-array n'=0,1,…,N2-1 form the second-level planar subarray, expressed as

[0130]

[0131] 4.2) The incident direction of the interference sound source (θ j' ,φ j' ) to widen the possible incident area Θ of the signal of the interfering sound source j' , for Θ j' The interference steering vector d2(θ j' ,φ j' ), j'=1,2,…,J, where J represents the number of interference angles after discretization, d2(θ j' ,φ j' ) is calculated according to formula (6); the correlation matrix of the discrete interference steering vector is constructed as follows:

[0132]

[0133] 4.3) Performing eigenvalue decomposition on the correlation matrix C yields λ1≥λ2≥…≥λ J , where λ k is the eigenvalue of the decomposition, v k is the corresponding eigenvector. The eigenvector with the largest eigenvalue is selected as the basis vector to form the projection matrix. Let the parameter K traverse from 1 to J and determine whether it satisfies the following inequality:

[0134]

[0135] Wherein, ε is a set constant, and in this embodiment, ε=0.01. If the inequality is satisfied, the traversal process is stopped, and K is determined to be the number of selected large eigenvalues.

[0136] 4.4) Select the eigenvectors corresponding to the first K largest eigenvalues ​​to form the projection matrix Use the projection matrix to preprocess the second-level plane subarray to obtain the projected subarray signal The covariance matrix of the projected subarray signal is:

[0137]

[0138] Where R is the covariance matrix of the second-level plane subarray y1, and N is the number of snapshots for calculating the signal covariance matrix sampling.

[0139] 4.5) The covariance matrix of the projected subarray signal obtained in step 4.4) and the known second-level planar subarray target signal steering vector d2(θ s ,φ s ), the zero-stretched linear constrained minimum variance LCMV adaptive weight vector of the second-level planar subarray is calculated as:

[0140]

[0141] Among them, d2(θ s ,φ s ) is calculated according to formula (6), Represents the inversion of the covariance matrix of the projected sub-array signal.

[0142] 4.6) If Figure 3 As shown, the second-level planar subarrays are weighted and summed to obtain the final beamforming output frequency domain signal

[0143] Step 5: Perform inverse Fourier transform on the final beamforming output frequency domain signal calculated in step 4 to restore the time domain signal of each frame. At the same time, continuously determine whether the direction of the target sound source and the interference sound source pointed by the beam has changed. If not, there is no need to update the weight coefficients, and repeatedly perform steps 1 to 4 to output the speech enhancement signal; if there is a change, only the subarray beamforming weight coefficients need to be updated according to steps 3 and 4, and then a new speech enhancement signal is output.

[0144] Figure 4 A comparison of the broadband beam response diagrams of the present embodiment and common beamforming methods with respect to frequency and angle is given, where Figure 4 (a), (b), (c), and (d) are broadband beam response diagrams of the delay-sum beamforming method, the windowed constant beamwidth beamforming method, the LCMV adaptive beamforming method, and the method of this embodiment, respectively. It can be seen that the main lobe width of the delay-sum beamforming method will become significantly wider as the frequency becomes lower, and when there is an azimuth deviation of the actual target signal, it will cause signal distortion and high-frequency component loss. In addition, the delay-sum beamforming method and the windowed constant beamwidth beamforming method cannot form null suppression at the specified interference angle. Although the LCMV adaptive beamforming method can form nulls at the specified horizontal angle directions of +25° and -30° (the horizontal angle in the method of this embodiment can be easily calculated from the pitch angle and azimuth angle), the nulls are relatively narrow and the constant main lobe width cannot be guaranteed. The beam of the method of this embodiment has a constant main lobe width over a wide frequency range, which can avoid distortion of the target signal and degradation of voice quality. At the same time, it can be seen that compared with the LCMV method, the method of this embodiment forms a deeper suppression at the interference angles of +25° and -30° horizontal angles and the area around the interference incident angle, solving the problem of deterioration of noise interference suppression caused by the accuracy deviation of the interference incident angle. At the same time, the method of this embodiment also has a greater sidelobe suppression degree in other non-interference sidelobe directions, ensuring that the beamforming method has strong noise reduction capability and robustness.

[0145] Figure 5This is a comparison of the signal output power diagram of the method of this embodiment and the common beamforming method as the incident angle changes. It can be seen that the traditional windowed sidelobe control fixed beamforming can obtain lower sidelobes, but cannot directly obtain the null sink in the specified interference direction. It is necessary to use software to solve the optimization problem, but the complexity is too high and cannot be applied in real time. The LCMV adaptive beamforming method can form null sink suppression in the interference direction, but for the case of a large number of planar array elements, the computational complexity of inverting the covariance matrix is ​​extremely high and the robustness is very poor. At the same time, it can be seen that the sidelobe suppression in the non-interference direction is relatively poor. The method of this embodiment combines the advantages of the two-dimensional Chebyshev windowed constant beam width and the linear constrained minimum variance beamforming method. The windowed fixed beam method can ensure a constant beam width while effectively controlling the sidelobe suppression. The average sidelobe suppression in the method of this embodiment is as high as 23.2dB. At the same time, the interference direction is widened to ensure that effective interference suppression can be achieved when there is an angular deviation in the interference direction. In the figure, when the angle deviation is ±2°, the interference suppression described in the method of this embodiment is improved by about 15dB compared with the LCMV method, which can significantly suppress directional interference noise and improve the target speech quality.

[0146] Example 2

[0147] The steps of a uniform planar array constant beamwidth linear constrained minimum variance beamforming method of this embodiment are the same as those of embodiment 1, with the main difference being that the planar array of this embodiment is composed of 70 omnidirectional microphones, where N=10, M=7, and the array element spacings are d x = 0.035 m and d y = 0.05 meters, the pitch angle and horizontal angle of the target sound source wave direction are θ = 15°, φ = 30°, the pitch angle and horizontal angle of the interference sound source wave direction are θ = 25°, φ = 135°, and the FFT length is N FFT =512, sampling frequency f s =16000Hz. Step 1 converts the collected time domain signal into the frequency domain. According to step 2, define and divide the two-level uniform planar subarrays and define the subarray dimension parameters. In this embodiment, three groups of beamformers with different subarray dimension parameters are designed: the first group of beamformers N2=2, M2=2 (N1=9, M1=6), the second group of beamformers N2=3, M2=3 (N1=8, M1=5) and the third group of beamformers N2=4, M2=3 (N1=7, M1=5). Divide several first-level subarrays according to step 3 and specify the main lobe widths of the x and y axes. and The two-dimensional Chebyshev window weight coefficients on the wide frequency band are designed, and the weighted sum of all sub-arrays is performed to obtain the frequency domain output signal of each sub-array. According to step 4, the linear constrained minimum variance beam with zero-steering broadening in the direction of the interference sound source is calculated to obtain the output waveforms of the three groups of beamformers.

[0148] Figure 6 The output signal waveforms of three groups of beamformers using the method of this embodiment are compared. Figure 6 (a)-(c) in the figure are output signal waveforms of the three groups of beamformers after signal processing by the method steps of this embodiment. It can be seen that the first group of beamformers (N2=2, M2=2) effectively suppresses directional interference signals, but due to the insufficient degree of freedom provided for the second-level subarray LCMV adaptive beamforming algorithm, the beam sidelobes are raised, especially causing white noise amplification in the low frequency band. When the second group of beamformers (N2=3, M2=3) is selected, relatively sufficient degrees of freedom are provided. Although there is a slight amplification of white noise, it can ensure high voice quality after signal processing. The third group of beamformers (N2=4, M2=3) can eliminate both directional interference and white noise very well. In summary, the selection of the subarray dimension parameters in step 2 will affect the interference and noise suppression effect of the method of this embodiment. Generally, the larger the dimension parameters N2 and M2 of the secondary subarray are, the better the noise reduction effect is. However, as N2 and M2 become larger, the improvement of the noise reduction effect is gradually weakened and the complexity is increased. Therefore, it is necessary to select appropriate parameters to balance the interference noise suppression degree and the calculation complexity of real-time implementation.

[0149] To verify the effectiveness of the method of this embodiment, the measured data experimental analysis was carried out using the above 70-element uniform array. Figure 7 Spectrograms of different methods when the parameters of this embodiment are set. Figure 7 (a) is the clean target speech, (b) is the noisy signal received by any array element, (c) is the constant beamwidth beamforming signal, (d) is the LCMV adaptive beamforming signal, and (e) is the signal of the method of this embodiment. Figure 7 From the results in , it can be seen that although the traditional constant beamwidth beamforming method can control the sidelobes, it cannot effectively suppress directional interference noise. There is a large amount of interference noise residual in the processed signal. The LCMV adaptive beamforming method can suppress directional interference relatively well, but due to the consistency between channels and the deviation of the sound source direction accuracy, the robustness is poor and it cannot completely and effectively suppress directional interference and white noise. It can be seen from the spectrogram that the low-frequency white noise is amplified. Its computational complexity also makes it impractical. The method in this embodiment has a good directional interference and white noise suppression effect and shows good robustness in actual measurements.

[0150] Example 3

[0151] This embodiment provides a uniform array constant beamwidth linear constrained minimum variance beamforming device for implementing the method of embodiment 1, such as Figure 8 As shown, it includes a planar array with uniformly distributed microphone array elements, a first-level planar sub-array signal processing module, a second-level planar sub-array signal processing module and a signal conversion processor.

[0152] The planar array is used to collect sound signals in the environment and convert each frame of sound signals into frequency domain signals;

[0153] The planar array is divided into a first-level planar sub-array and a second-level planar sub-array, the first-level planar sub-array signal processing module is used to perform beamforming on the first-level planar sub-array, and the second-level planar sub-array signal processing module is used to process the frequency domain signal output by the first-level planar sub-array signal processing module and output a final beamforming output frequency domain signal;

[0154] The signal conversion processor is used to convert the beamforming output frequency domain signal into a time-frequency signal.

[0155] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all of these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims

1. A uniform array constant beamwidth linearly constrained minimum variance beamforming method, characterized in that: The steps include: Step 1: A planar array with evenly distributed microphone elements collects sound signals in the environment, and converts each frame of sound signals collected by the planar array into a frequency domain through short-time Fourier transform processing; Step 2: The planar array is regarded as a cascade representation of two-level planar sub-arrays, and the weight matrix of the planar array is represented as a two-dimensional convolution of the weight matrices of the two-level planar sub-arrays; the dimension parameters of the two-level planar sub-arrays are set respectively to obtain the corresponding sub-array steering matrix and sub-array steering vector; Step 3, dividing the first-level planar subarray: select a first-level planar subarray on the planar array, obtain a number of overlapping first-level planar subarrays by translation, and use a two-dimensional Chebyshev windowed constant beamwidth beamforming method to perform weighted summation on the frequency domain signal of each first-level planar subarray to obtain the frequency domain output of each first-level planar subarray; Step 4: The output signals of each first-stage planar subarray are combined into a second-stage planar subarray, a linear constrained minimum variance beamformer with null widening is constructed, and adaptive spatial filtering is performed to enhance the target direction signal and form wide null suppression in the interference direction; Step 5: Perform inverse Fourier transform on the final beamforming output frequency domain signal calculated in step 4 to restore the time domain signal of each frame; at the same time, continuously determine whether the wave directions of the target sound source and the interference sound source pointed by the beam have changed. If they have not changed, there is no need to update the weight coefficients, and repeatedly perform steps 1 to 4 to output the speech enhancement signal; If a change occurs, it is only necessary to update the subarray beamforming weight coefficients according to steps 3 and 4, and then output a new speech enhancement signal.

2. The method according to claim 1, characterized in that The specific process of step 1 is: The sound signal is collected by using a planar array with uniformly distributed microphone elements in the spatial topology. The microphone elements are uniformly distributed on the x-axis and y-axis two-dimensional planes, and are composed of M×N omnidirectional microphones. M is the number of rows of the planar array in the y-axis direction, N is the number of columns of the planar array in the x-axis direction, and the spacing between the microphone elements on the x-axis and y-axis is δ x and δ y ; Taking the microphone array element at the upper left corner of the plane array as the reference point and the origin of the coordinate system, each microphone array element can be represented by a serial number (m, n), m = 0, 1, ..., M-1, n = 0, 1, ..., N-1, where m and n represent the serial numbers of the row and column of the microphone array element in the plane array, respectively. The spatial position coordinates of the microphone array element (m, n) in the coordinate system can be expressed as (nδ x ,mδ y ,0), the time domain signal collected by the microphone array element (m,n) is transformed into the frequency domain form X by short-time Fourier transform mn (k), k is the frequency point f k The serial number, f k =kf s / N FFT , f s is the sampling rate, N FFT is the Fourier transform length; considering the far-field acoustic model, θ and φ are used to represent the elevation angle and azimuth angle of the incoming wave direction of the sound source, respectively, and the steering matrix of the M×N planar array is defined as: where D(f,θ,φ) is the array steering matrix, [D(f,θ,φ)] mn represents the element in the mth row and nth column of the array steering matrix, the wave number parameter c is the propagation speed of sound in the environment, and f is the frequency; the columns of the array steering matrix are stacked as a vector representation to obtain the corresponding array steering vector:

3. The method according to claim 2, characterized in that Planar array beamforming is a spatial linear filtering of the array frequency domain signal. The spatial filtering weight coefficients of the planar array are represented by the weight matrix W: The weight matrix is ​​stacked column by column as a vector representation, resulting in a flat array of weight vectors: in=[in 00 ,…,In m0 ,…,In (M-1)0 ,…,In mn ,…,In (M-1)(N-1) ] T (4) The beam pattern corresponding to the weight vector of the planar array is B(θ,φ,w)=d H (θ, φ)w, the planar array is regarded as a cascade of two-stage planar sub-arrays, and its beam pattern is equivalent to the multiplication of two sub-beam patterns B(θ, φ, w) = B1(θ, φ, w1) B2(θ, φ, w2), and a fixed or adaptive sub-array beamformer is designed separately; the weight vectors of the two-stage planar sub-arrays are w1 and w2 respectively; the steering vectors of the two-stage planar sub-arrays are: Where M1 and N1 are the number of rows and columns of the first-level planar subarray, M2 and N2 are the number of rows and columns of the second-level planar subarray, and the beam pattern is expressed in the form of a Cauchy product: Among them, w mn is the weight coefficient of the microphone array element (m,n), and is the weight coefficient of the corresponding element of the two-level planar subarray. To simplify the expression, and According to formula (5), assuming that the number of rows and columns of the planar array and the two-level planar sub-array satisfy the relationship M = M1 + M2-1 and N = N1 + N2-1, the weight matrix of the planar array is expressed as the two-dimensional convolution of the weight matrix of the two-level planar sub-array: Among them, parameters a1 = max(0, m-1 + M2), a2 = min(m, M1-1), b1 = max(0, n-1 + N2) and b2 = min(n, N2-1).

4. The method according to claim 3, characterized in that The row and column parameters of the first-level planar subarray are M1=6, N1=8, and the first-level planar subarray is a 6×8 planar array. The corresponding row and column parameters of the second-level planar subarray can be calculated as M2=M+1-M1=3, N2=N+1-N1=3, and the second-level planar subarray is a 3×3 planar array.

5. The method according to claim 3, characterized in that: The specific process of step 3 is: 3.1) Divide the first-level plane subarray: Using the invariant property of the plane array formation, according to the row parameter M1 and column parameter N1 set in step 2, select consecutive N1 columns in consecutive M1 rows to form an M1×N1 subarray. By translation on the M×N plane array, (M-M1+1)(N-N1+1)=M2N2 first-level plane subarrays are obtained. Each first-level plane subarray is represented by a serial number (m',n')m'=0,1,...,M2-1,n'=0,1,...,N2-1, where n' and m' represent the serial numbers of the row and column of each first-level plane subarray, respectively. Then, the frequency domain representation of the (m,n)th microphone array element in the first-level plane subarray (m',n') is X (m'+m)(n'+n) , where m = 0, 1, ..., M1-1, n = 0, 1, ..., N1-1; 3.2) Calculate the Chebyshev coefficients of each first-level planar subarray in the x-axis direction, determine the x-axis main lobe width according to the design goal, and calculate the approximate intermediate value Get the Chebyshev weighted beam zero position: Construct the beam zero position steering vector matrix: Among them, the steering vector represented by each column in the matrix is: Then the Chebyshev coefficient in the x-axis direction is expressed as where e1 = [1, 0, ..., 0] T ; 3.3) According to the design target y-axis main lobe width Calculate the Chebyshev coefficient h of each first-level plane subarray in the y-axis direction (y) : Calculates approximate middle value Get the beam zero point position of each first-level planar subarray in the y-axis dimension: Construct the beam zero position steering vector matrix: Among them, the steering vector represented by each column in the matrix is: Then the Chebyshev coefficient of each first-level plane subarray in the y-axis direction can be expressed as 3.4) The Chebyshev coefficient vector h of each first-level plane subarray is given by h (x) and h (y) Together, the elements in the vector are m=0,1,…,M1-1,n=0,1,…,N1-1,where and They are vector h (x) and h (y) The elements in; the weight vector w1 of the two-dimensional Chebyshev windowed beamformer is: w1=diag{h}d1(θ s ,f s )=Λd1(θ s ,f s ) (10) Among them, θ s ,φ s are the pitch angle and horizontal angle of the target sound source s, respectively, s ,φ s ) is calculated according to formula (5), Λ=diag{h} represents a diagonal matrix with vector h as the matrix diagonal element, and Chebyshev windowed beamforming is used to achieve effective control of side lobes while keeping the beam width unchanged in the frequency domain; 3.5) The frequency domain signal of each first-level planar subarray is spatially filtered using the corresponding two-dimensional Chebyshev windowed beamforming weight vector, and the output signal of each first-level planar subarray (m', n') is obtained as follows: in, is the weight vector The corresponding element in (·) * denotes the conjugation of the elements in brackets.

6. The method according to claim 5, characterized in that The specific process of step 4 is: 4.1) The output signal of the first-level planar sub-array m'=0,1,…,M2-1,n'=0,1,…,N2-1compose the second-level planar subarray, which is expressed as 4.2) The incident direction of the interference sound source (θ j' ,φ j' ) to widen the possible incident area Θ of the signal of the interfering sound source j' , for Θ j' The interference steering vector d2(θ j' ,φ j' ), j'=1,2,…,J, where J represents the number of interference angles after discretization, d2(θ j' ,φ j' ) is calculated according to formula (6); the correlation matrix of the discrete interference steering vector is constructed as follows: 4.3) Performing eigenvalue decomposition on the correlation matrix C yields λ1≥λ2≥…≥λ J , where λ k is the eigenvalue of the decomposition, v k is the corresponding eigenvector. The eigenvector with the larger eigenvalue is selected as the basis vector to form the projection matrix. Let the parameter K traverse from 1 to J and determine whether it satisfies the following inequality: Among them, ε is a set constant. If the inequality is satisfied, the traversal process is stopped, and K is determined to be the number of selected large eigenvalues; 4.4) Select the eigenvectors corresponding to the first K largest eigenvalues ​​to form the projection matrix Use the projection matrix to preprocess the second-level plane subarray to obtain the projected subarray signal The covariance matrix of the projected subarray signal is: Where R is the covariance matrix of the second-level plane subarray y1, and N is the number of snapshots for calculating the signal covariance matrix sampling; 4.5) The covariance matrix of the projected subarray signal obtained in step 4.4) and the known second-level planar subarray target signal steering vector d2(θ s ,φ s ), the zero-stretched linear constrained minimum variance LCMV adaptive weight vector of the second-level planar subarray is calculated as: Among them, d2(θ s ,φ s ) is calculated according to formula (6), represents the inversion of the covariance matrix of the projected sub-array signal; 4.6) Perform weighted summation on the second-level planar subarray to obtain the final beamforming output frequency domain signal 7. A uniform array constant beamwidth linearly constrained minimum variance beamforming device for implementing any of the methods described in claims 1-6, characterized in that: It includes a planar array with uniformly distributed microphone elements, a first-level planar sub-array signal processing module, a second-level planar sub-array signal processing module and a signal conversion processor; The planar array is used to collect sound signals in the environment and convert each frame of sound signals into frequency domain signals; The planar array is divided into a plurality of first-level planar sub-arrays, the first-level planar sub-array signal processing module is used to perform beamforming on each first-level planar sub-array, and the second-level planar sub-array signal processing module is used to perform beamforming on the frequency domain signal output by the first-level planar sub-array signal processing module to obtain a final beamforming output frequency domain signal; The signal conversion processor is used to convert the beamforming output frequency domain signal into a time-frequency signal.

Images