A beamforming method and device based on room environment adaptive calibration
By setting up the microphone array and testing sound source in the room environment, calculating the reflected sound coefficient and building an optimized equation system, the problem of low signal-to-noise ratio of the microphone array in the room environment is solved, and a more efficient sound signal reception effect is achieved.
Patent Information
- Application Number
- CN202410447704.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-15
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-04-15
AI Technical Summary
In a room environment, the acoustic signals received by the microphone array contain coherent interference of direct acoustic signals and reflected acoustic signals, resulting in a decrease in the signal-to-noise ratio. The existing adaptive beamforming technology cannot effectively utilize the acoustic signal energy in the room environment.
By setting the microphone array and testing the sound source in the room, obtaining the sound signal matrix, calculating the array covariance matrix, estimating the reflected sound coefficient, building a set of optimization equations with constraints, calculating the optimal weighting vector, and weighting the sound signal matrix to achieve self-importance Adapt to beamforming.
By constraining the orthogonal equations of the reflected sound coefficient and the signal subspace, the optimal weighted vector is calculated, and the coherence addition of the direct sound and reflected sound of the microphone array is realized, the directional gain of signal reception is improved, and the acoustic signal energy is fully utilized.
Smart Images

Figure CN118197341B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of beamforming, and in particular to a beamforming method and device based on room environment adaptive calibration. Background Art
[0002] Beamforming is the combination of antenna technology and digital signal processing technology for the purpose of directional signal transmission or reception.
[0003] When the microphone array receives sound signals in a room, the received sound signals include not only direct sound signals but also reflected sound signals. The direct sound signals and the reflected sound signals are mutually coherent signals, and the reflected sound signals will interfere with the desired signals, resulting in a decrease in the signal-to-noise ratio of the signal obtained by beamforming.
[0004] The existing adaptive calibrated beamforming technology mainly controls the sidelobe gain to reduce the receiving weight of the microphone array for the reflected sound signal. Although it increases the signal-to-noise ratio of the obtained sound signal, in a room environment, the intensity of the reflected sound signal is equivalent to that of the direct sound signal. Therefore, the existing adaptive beamforming method will cause most of the sound signal energy received by the microphone array in a room environment to not be effectively utilized, and the ideal effect cannot be achieved. Summary of the invention
[0005] In view of this, an object of the present invention is to provide a beamforming method and device based on room environment adaptive calibration to solve the technical problem of poor acoustic signal reception in a room environment.
[0006] The present invention provides a beamforming method based on room environment adaptive calibration, comprising:
[0007] S1: By setting a microphone array and multiple test sound sources in the room environment, the sound signal matrix X(t) in the room environment is obtained;
[0008] S2: Calculate the array covariance matrix R through the acoustic signal matrix X(t) X , get the array covariance matrix R X The noise subspace U in N and signal subspace U s ;
[0009] S3: Through the noise subspace U N and signal subspace U s Estimate the reflected sound coefficient ρ;
[0010] S4: Construct a constrained optimization equation set through the reflected sound coefficient ρ, and calculate the optimal weight vector W through the constrained optimization equation set opt ;
[0011] S5: Through the optimal weight vector W opt The acoustic signal matrix X(t) is weighted to obtain the adaptive beamforming output X(t) of the microphone array opt .
[0012] Preferably, step S1 specifically comprises:
[0013] S11: Place a microphone array inside the room environment and place multiple test sound sources around the microphone array;
[0014] S12: Each test sound source plays a test sound source transmission signal to the microphone array. After the microphone array obtains the test sound source transmission signal matrix S(t), it calculates and obtains the sound signal matrix X(t), where t represents time.
[0015] Preferably, the array covariance matrix R X The expression is:
[0016] R X =E[X(t)X H (t)]
[0017] =U s Λ s U s H +U N Λ N U N H
[0018] Where t represents time, E[·] represents the mean operation, [·] H represents the conjugate transpose operation of the matrix, Λ s is the array covariance matrix R X The diagonal matrix composed of large eigenvalues of s is the array covariance matrix R X The signal subspace composed of eigenvectors with large eigenvalues, Λ N is the array covariance matrix R X The diagonal matrix composed of small eigenvalues of N is the array covariance matrix R X The noise subspace is composed of eigenvectors with small eigenvalues.
[0019] Preferably, step S3 specifically comprises:
[0020] S31: According to the orthogonality criterion, construct the noise subspace U N and signal subspace U s The orthogonal equations are:
[0021]
[0022] where ||·|| represents the Frobenius norm, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array;
[0023] S32: Substitute the constraints into the orthogonal equations to obtain the optimized orthogonal equations. The expression of the optimized orthogonal equations is:
[0024]
[0025] in, represents the minimum objective function with ρ as the variable, st represents the constraint condition, k=[1,0] T Represents a 2×1 dimensional vector;
[0026] S33: The reflected sound coefficient ρ is obtained by calculating the optimized orthogonal equation group. The expression of the reflected sound coefficient ρ is:
[0027]
[0028] in,(·) -1 represents the inverse matrix, and y=[0,1] represents a 2×1 dimensional vector.
[0029] Preferably, step S4 is specifically:
[0030] S41: Obtain the acoustic signal gain in the direct sound direction of the microphone array and the gain of the reflected sound direction of the microphone array Among them, W is the initial weight vector, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array;
[0031] S42: Yes and Constraints are imposed and a constrained optimization equation set is constructed. The expression of the constrained optimization equation set is:
[0032]
[0033] in, represents the minimum objective function with W as the variable, and st represents the constraint condition;
[0034] S43: Solve the initial weight vector W in the constrained optimization equations by Lagrange multiplier method to obtain the optimal weight vector W opt , W opt The expression is:
[0035]
[0036] in,(·) -1 represents the inverse matrix, (·) T Represents the transpose of a matrix.
[0037] Preferably, in step S5, the adaptive beamforming output X(t) opt The expression is:
[0038] X(t) opt =W opt H X(t)=2S(t)+W opt H N(t)
[0039] Wherein, S(t) is the test sound source transmission signal matrix, N(t) is the Gaussian white noise matrix of the microphone array, and t represents time.
[0040] A storage medium stores instructions and data for implementing the beamforming method based on room environment adaptive calibration.
[0041] A beamforming device based on adaptive calibration of room environment comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement the beamforming method based on adaptive calibration of room environment.
[0042] The present invention has the following beneficial effects:
[0043] The reflected sound coefficient is obtained by constraining the calculation of the orthogonal equation group constructed by the noise subspace and the signal subspace, so that the reflected sound coefficient is constrained to the minimum value, so that the adaptive beamforming output has a higher directional gain in the target direction, and the echo signal can be better received; the optimal weighting vector is calculated by the constrained optimization equation group constructed by the reflected sound coefficient, and the sound signal matrix is weighted by the optimal weighting vector, so that the sound signal in the direct sound direction of the microphone array and the sound signal in the reflected sound direction of the microphone array are coherently added, which makes full use of the sound signal energy received by the microphone array and can effectively solve the problem of sound signal reception in the room environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;
[0045] Figure 2 The figure is a schematic diagram of the working of the hardware device according to an embodiment of the present invention.
[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0047] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0048] Reference Figure 1 The present invention provides a beamforming method based on room environment adaptive calibration, comprising:
[0049] S1: By setting a microphone array and multiple test sound sources in the room environment, the sound signal matrix X(t) in the room environment is obtained;
[0050] Specifically, multiple test sound sources are set at different angles in the room environment, and the method for obtaining the sound signal matrix in the room environment is as follows:
[0051] The microphone array performs spatial sampling in the room environment. The collected sound signal contains the spatial information of the room environment. The digital signal data sampled by the microphone is output to obtain the sound signal matrix.
[0052] Step S1 is specifically as follows:
[0053] S11: Place a microphone array inside the room environment and place multiple test sound sources around the microphone array;
[0054] S12: Each test sound source plays a test sound source transmission signal to the microphone array. After the microphone array obtains the test sound source transmission signal matrix S(t), it calculates and obtains the sound signal matrix X(t), where t represents time.
[0055] Specifically, the expression of the acoustic signal matrix obtained by the microphone array is:
[0056]
[0057] in,[·] T represents the transpose operation of the matrix, X(t)=[x1(t),x2(t),…,x N (t)] T is the acoustic signal matrix obtained by the microphone array, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array.
[0058] S2: Calculate the array covariance matrix R through the acoustic signal matrix X(t) X , get the array covariance matrix R X The noise subspace U in N and signal subspace U s ;
[0059] Specifically, the method of obtaining the noise subspace and the acoustic signal space in the array covariance matrix is:
[0060] The acoustic signal matrix acquired by the microphone array is output to the digital language processing module, and the digital language processing module calculates the acoustic signal matrix by directly connecting to the output end of the microphone array.
[0061] Array covariance matrix R X The expression is:
[0062] R X =E[X(t)X H (t)]
[0063] =U s Λ s U s H +U N Λ N U N H
[0064] Where t represents time, E[·] represents the mean operation, [·] H represents the conjugate transpose operation of the matrix, Λ s is the array covariance matrix R X The diagonal matrix composed of large eigenvalues of s is the array covariance matrix R X The signal subspace composed of eigenvectors with large eigenvalues, Λ N is the array covariance matrix R X The diagonal matrix composed of small eigenvalues of N is the array covariance matrix R X The noise subspace is composed of eigenvectors with small eigenvalues.
[0065] Specifically, the array covariance matrix R X The expression can be further expanded as follows:
[0066]
[0067] Among them, σ N is the variance of Gaussian white noise, I N Represents the N×N-dimensional identity matrix.
[0068] S3: Through the noise subspace U N and signal subspace U s Estimate the reflected sound coefficient ρ;
[0069] Step S3 is specifically as follows:
[0070] S31: According to the orthogonality criterion, construct the noise subspace U N and signal subspace U s The orthogonal equations are:
[0071]
[0072] where ||·|| represents the Frobenius norm, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array;
[0073] S32: Substitute the constraints into the orthogonal equations to obtain the optimized orthogonal equations. The expression of the optimized orthogonal equations is:
[0074]
[0075] in, represents the minimum objective function with ρ as the variable, st represents the constraint condition, k=[1,0] T Represents a 2×1 dimensional vector;
[0076] S33: The reflected sound coefficient ρ is obtained by calculating the optimized orthogonal equation group. The expression of the reflected sound coefficient ρ is:
[0077]
[0078] in,(·) -1 represents the inverse matrix, and y=[0,1] represents a 2×1 dimensional vector.
[0079] S4: Construct a constrained optimization equation set through the reflected sound coefficient ρ, and calculate the optimal weight vector W through the constrained optimization equation set opt ;
[0080] The acoustic signal matrix X(t) obtained by the microphone array is weighted by the initial weight vector W to obtain the initial weighted output X(t) of the microphone array. W :
[0081]
[0082] in, is the acoustic signal gain in the direct sound direction of the microphone array, is the acoustic signal component in the direct sound direction of the microphone array, is the acoustic signal gain in the direction of the reflected sound from the microphone array, The acoustic signal component in the direction of the reflected sound from the microphone array;
[0083] Through and The optimal weight vector W can be solved by applying constraints opt , as shown in step S4;
[0084] Step S4 is specifically as follows:
[0085] S41: Obtain the acoustic signal gain in the direct sound direction of the microphone array and the gain of the reflected sound direction of the microphone array Among them, W is the initial weight vector, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array;
[0086] S42: Yes and Constraints are imposed and a constrained optimization equation set is constructed. The expression of the constrained optimization equation set is:
[0087]
[0088] in, represents the minimum objective function with W as the variable, and st represents the constraint condition;
[0089] S43: Solve the initial weight vector W in the constrained optimization equations by Lagrange multiplier method to obtain the optimal weight vector W opt , W opt The expression is:
[0090]
[0091] in,(·) -1 represents the inverse matrix, (·) T Represents the transpose of a matrix.
[0092] S5: Through the optimal weight vector W opt The acoustic signal matrix X(t) is weighted to obtain the adaptive beamforming output X(t) of the microphone array opt .
[0093] The adaptive beamforming output X(t) in step S5 is opt The expression is:
[0094] X(t) opt =W opt H X(t)=2S(t)+W opt H N(t)
[0095] Wherein, S(t) is the test sound source transmission signal matrix, N(t) is the Gaussian white noise matrix of the microphone array, and t represents time.
[0096] Specifically, the adaptive beamforming output X(t) opt The expression can be further expanded as follows:
[0097]
[0098] From the above formula, we can get the acoustic signal gain in the direct sound direction of the microphone array: and the gain of the reflected sound direction of the microphone array Constraints are imposed and weighted calculation is performed on the acoustic signal matrix X(t) obtained by the microphone array, so that the acoustic signal in the direct sound direction of the microphone array and the acoustic signal in the reflected sound direction of the microphone array are coherently added, making full use of the acoustic signal energy received by the microphone array, which can effectively solve the problem of acoustic signal reception in a room environment.
[0099] See also Figure 2 , Figure 2 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically comprises: a beamforming device 401 based on adaptive calibration of room environment, a processor 402 and a storage medium 403.
[0100] A beamforming device 401 based on adaptive calibration of room environment: The beamforming device 401 based on adaptive calibration of room environment implements the beamforming method based on adaptive calibration of room environment.
[0101] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the beamforming method based on adaptive calibration of room environment.
[0102] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the beamforming method based on adaptive calibration of room environment.
[0103] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0104] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.
[0105] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A beamforming method based on room environment adaptive calibration, characterized in that: include: S1: By setting a microphone array and multiple test sound sources in the room environment, the sound signal matrix X(t) in the room environment is obtained; S2: Calculate the array covariance matrix R through the acoustic signal matrix X(t) X , get the array covariance matrix R X The noise subspace U in N and signal subspace U s ; S3: Through the noise subspace U N and signal subspace U s Estimate the reflected sound coefficient ρ; S4: Construct a constrained optimization equation set through the reflected sound coefficient ρ, and calculate the optimal weight vector W through the constrained optimization equation set opt ; S5: Through the optimal weight vector W opt The acoustic signal matrix X(t) is weighted to obtain the adaptive beamforming output X(t) of the microphone array opt ; Step S3 is specifically as follows: S31: According to the orthogonality criterion, construct the noise subspace U N and signal subspace U s The orthogonal equations are: where ||·|| represents the Frobenius norm, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array; S32: Substitute the constraints into the orthogonal equations to obtain the optimized orthogonal equations. The expression of the optimized orthogonal equations is: in, represents the minimum objective function with ρ as the variable, st represents the constraint condition, k=[1,0] T Represents a 2×1 dimensional vector; S33: The reflected sound coefficient ρ is obtained by calculating the optimized orthogonal equation group. The expression of the reflected sound coefficient ρ is: in,(·) -1 represents the inverse matrix, y=[0,1] represents a 2×1 dimensional vector, [·] H Represents the conjugate transpose operation of a matrix.
2. The beamforming method based on room environment adaptive calibration according to claim 1, characterized in that: Step S1 is specifically as follows: S11: Place a microphone array inside the room environment and place multiple test sound sources around the microphone array; S12: Each test sound source plays a test sound source transmission signal to the microphone array. After the microphone array obtains the test sound source transmission signal matrix S(t), it calculates and obtains the sound signal matrix X(t), where t represents time.
3. The beamforming method based on room environment adaptive calibration according to claim 1, characterized in that: Array covariance matrix R X The expression is: R X =E[X(t)X H (t)] =U s Λ s IN s H +U N Λ N IN N H Where t represents time, E[·] represents the mean operation, and Λ s is the array covariance matrix R X The diagonal matrix composed of large eigenvalues of s is the array covariance matrix R X The signal subspace composed of eigenvectors with large eigenvalues, Λ N is the array covariance matrix R X The diagonal matrix composed of small eigenvalues of N is the array covariance matrix R X The noise subspace is composed of eigenvectors with small eigenvalues.
4. The beamforming method based on room environment adaptive calibration according to claim 1, characterized in that: Step S4 is specifically as follows: S41: Obtain the acoustic signal gain in the direct sound direction of the microphone array and the gain of the reflected sound direction of the microphone array Among them, W is the initial weight vector, is the direct sound direction steering vector of the microphone array, is the direction steering vector of the reflected sound from the microphone array; S42: Yes and Constraints are imposed and a constrained optimization equation set is constructed. The expression of the constrained optimization equation set is: in, represents the minimum objective function with W as the variable, and st represents the constraint condition; S43: Solve the initial weight vector W in the constrained optimization equations by Lagrange multiplier method to obtain the optimal weight vector W opt , W opt The expression is: in,(·) -1 represents the inverse matrix, (·) T Represents the transpose of a matrix.
5. The beamforming method based on room environment adaptive calibration according to claim 4, characterized in that: The adaptive beamforming output X(t) in step S5 is opt The expression is: X(t) opt =W opt H X(t)=2S(t)+W opt H N(t) Wherein, S(t) is the test sound source transmission signal matrix, N(t) is the Gaussian white noise matrix of the microphone array, and t represents time.
6. A storage medium, characterized in that: The storage medium stores instructions and data for implementing the beamforming method based on room environment adaptive calibration as described in any one of claims 1 to 5.
7. A beamforming device based on room environment adaptive calibration, characterized in that: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement the beamforming method based on room environment adaptive calibration as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Adaptive coherent beamforming method based on multipath information
CN109633563A
Adaptive coherent beam formation method of multi-path utilization
CN109901132A
Enhanced Beamforming for Arrays of Directional Microphones
US20080240463A1
Signal processing device, signal processing method, and program
WO2020121545A1
