Speech enhancement method, device and equipment
Through the combination of diffusion sound field model and post-filter, the problem of poor anti-interference ability of sound pickup algorithm in complex environments and the signal-to-noise ratio decreases when the distance is large, and the high flexibility in complex environments is achieved Strong and high signal-to-noise ratio voice signal enhancement effect.
Patent Information
- Application Number
- CN202011592688.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-12-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2040-12-29
AI Technical Summary
The existing sound pickup algorithm has poor anti-interference capability in complex environments, and the signal-to-noise ratio of the voice signal decreases when the distance between the sound source and the sound pickup device is large, which affects the use effect.
The diffusion sound field model is used for beamforming, and the weight vector of the beamformer is determined through the coordinates of the microphone array and the arrival angle of the speech signal, and the data processing is carried out in combination with the post-filter to achieve enhanced speech signal without relying on noise characteristic estimation or first Verify information.
It improves the robustness and reliability of speech signals in complex environments, effectively suppresses noise interference, enhances the clarity of target speech, and maintains the continuity of speech signals when multiple people speak or the environment is noisy.
Smart Images

Figure CN112712818B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of acoustics, and in particular relates to a speech enhancement method, device and equipment. Background Art
[0002] With the development of artificial intelligence, more and more living environments will be more intelligent, such as in-vehicle environments, home environments, classroom environments, conference room environments, etc. A variety of intelligent devices are usually used in these environments, such as smart interactive devices used at home, or sound pickup devices used in classrooms and conference rooms. Whether it is an interactive device or a sound pickup device, the most important thing is to accurately identify the sound source.
[0003] There are many existing sound pickup algorithms, and one of the most widely used is the spherical microphone array sound pickup algorithm. The spherical microphone array sound pickup algorithm is based on the principle of beamforming, which picks up the desired voice signal in a complex environment, that is, only the sound signal propagating from the desired direction is picked up, while the noise and interference signals from other directions will be suppressed, thereby achieving the effect of target voice enhancement.
[0004] However, existing sound pickup algorithms have certain problems in the process of picking up voice signals, such as poor anti-interference ability and excessive reverberation in complex environments. In addition, when the distance between the sound source and the sound pickup device is large, the signal-to-noise ratio of the voice signal picked up by the existing sound pickup algorithm will drop significantly, thereby sharply reducing the intelligibility of the picked up voice signal, affecting the use effect of the sound pickup device. Summary of the invention
[0005] In order to solve the problems of the prior art, the present invention proposes a speech enhancement method, device and apparatus. The method can achieve target speech enhancement without estimating the characteristics of noise and without relying on any prior information, and has strong robustness and high reliability.
[0006] The specific technical solutions provided by the embodiments of the present invention are as follows:
[0007] In a first aspect, a speech enhancement method is provided, the method comprising:
[0008] Preprocessing the speech signal containing the target speech collected by the microphone array to obtain time-frequency spectrum data of the speech signal;
[0009] Based on a pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal, a weight vector of a beamformer is determined, and beamforming is performed on the time-frequency spectrum data using the weight vector; wherein the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field condition is met;
[0010] Filtering the beamformed data using a post filter;
[0011] The filtered data is transformed to obtain the enhanced target speech.
[0012] In some embodiments, the post-filter is any one of a minimum mean square error estimator, a log spectrum amplitude estimator, a short-time spectrum amplitude estimator and a Zelinski filter.
[0013] In some embodiments, the post filter is a Wiener filter, and filtering the beamformed data using the post filter specifically includes:
[0014] Calculating correlation coefficients between microphone channels based on the diffuse sound field model;
[0015] The post-filter coefficient of the Wiener filter is calculated by using the correlation coefficient between the microphone channels, the estimated value of the auto-power spectrum of the speech in the speech signal, and the estimated value of the auto-power spectrum of the noise;
[0016] Performing Wiener filtering on the beamformed data according to the post-filter coefficients;
[0017] The estimated value of the auto-power spectrum of speech and the estimated value of the auto-power spectrum of noise in the speech signal are determined based on the time-frequency spectrum data and the correlation coefficient calculated according to a given calculation method.
[0018] In some embodiments, before performing Wiener filtering on the beamformed data according to the post-filter coefficients, the method further includes:
[0019] The post-filter coefficients are smoothed and median filtered.
[0020] In some embodiments, determining the weight vector of the beamformer based on the pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal specifically includes:
[0021] The diffuse sound field model is determined as the correlation matrix R N ;
[0022] The steering vector a is determined based on the sound wave frequency and sound speed of the speech signal, the coordinate matrix formed by the coordinates of each microphone array element, and the arrival angle of the speech signal. s ;
[0023] According to the correlation matrix R N , steering vector a s , the beamformer weight vector W is calculated using the preset calculation model opt ;
[0024] Wherein, the calculation model is:
[0025] In some embodiments, the method for constructing the diffuse sound field model includes:
[0026] Determine the relative distance between each microphone array element according to the coordinates of all microphone array elements;
[0027] The diffuse sound field matrix corresponding to each frequency point is determined based on the relative distance between each microphone array element, the sound wave frequency and the sound speed of the speech signal;
[0028] The diffuse sound field matrices corresponding to all frequency points are combined to form the diffuse sound field model.
[0029] In some embodiments, the preprocessing of the speech signal containing the target speech collected by the microphone array specifically includes:
[0030] The speech signal containing the target speech collected by the microphone array is framed, windowed and Fourier transformed.
[0031] In some embodiments, transforming the filtered data to obtain the enhanced target speech specifically includes:
[0032] Perform inverse Fourier transform on the filtered data;
[0033] The data obtained after inverse Fourier transformation is reconstructed using the stack-add method to obtain the enhanced target speech.
[0034] In a second aspect, a speech enhancement device is provided, the device comprising:
[0035] A data processing unit, used to pre-process the speech signal containing the target speech collected by the microphone array to obtain time-frequency spectrum data of the speech signal;
[0036] A beamforming unit, configured to determine a weight vector of a beamformer based on a pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal, and to perform beamforming on the time-frequency spectrum data using the weight vector; wherein the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field condition is met;
[0037] A post-filtering unit, used for filtering the beamformed data using a post-filter;
[0038] The data processing unit is also used to perform transformation processing on the filtered data to obtain enhanced target speech.
[0039] According to a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to the first aspect when executing the computer program.
[0040] The embodiments of the present invention have the following beneficial effects:
[0041] 1. The present invention determines the weight vector of the beamformer through the diffuse sound field model, the coordinates of each microphone array element in the microphone array and the arrival angle of the speech signal to perform beamforming and realize sound source recognition. Since the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field condition is met, the present invention is unlike traditional algorithms (such as methods based on spectral subtraction, Wiener filtering and adaptive enhancement, which require the characteristic estimation of noise, usually rely on the speech endpoint detection algorithm to estimate in the non-speech segment or use a reference microphone), the present method does not rely on any prior knowledge, can be used at any time, and has strong robustness and high reliability in complex environments;
[0042] 2. The present invention calculates the correlation coefficient between each microphone channel through a diffuse sound field model, and obtains the post-filter coefficient by combining the estimated value of the auto-power spectrum of the speech and noise in the speech signal, thereby performing Wiener filtering, which can further effectively enhance the speech in the target direction, effectively suppress other voice interference when multiple people are speaking, and also effectively suppress environmental noise interference;
[0043] 3. The present invention can make the reconstructed speech have good continuity and avoid the phenomenon of freezing by smoothing and median filtering the post-filter coefficients;
[0044] 4. The method described in the present invention is easy to integrate on video conferencing equipment, and the algorithm complexity is low, and it is easy to run in real time on an embedded platform with limited computing power. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0046] Figure 1 is an exemplary system architecture diagram to which some embodiments of the present application may be applied;
[0047] Figure 2 is a flow chart of an embodiment of a speech enhancement method according to the present application;
[0048] Figure 3is a flow chart of another embodiment of the speech enhancement method according to the present application;
[0049] Figure 4 is a flow chart of another embodiment of the speech enhancement method according to the present application;
[0050] Figure 5 is a structural schematic diagram of an embodiment of a speech enhancement device according to the present application;
[0051] Figure 6 It is a schematic diagram of the structure of a computer device suitable for implementing some embodiments of the present application. DETAILED DESCRIPTION
[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0053] As described in the background technology, in some more complex environmental scenarios, such as when there are multiple sound sources / the environmental noise is large, the traditional spherical microphone array pickup algorithm based on the beamforming principle has a weak anti-interference ability; in addition, when the sound source is far away from the pickup device, it is impossible to obtain an effective signal, resulting in a sharp drop in the intelligibility of the picked up voice signal. Based on this, the applicant of the present application creatively thought of using the principle of diffuse sound field to construct a diffuse sound field model, thereby performing beamforming processing, and then obtaining the target speech.
[0054] Among them, the diffuse sound field is an ideal sound field theoretically defined in the acoustic principle, which is defined as "a sound field with uniform energy density and random distribution in all propagation directions". It contains the following characteristics:
[0055] (1) The sound energy density at each point in the diffuse sound field is uniform; (2) The sound intensity reaching a point from all directions is approximately equal; (3) The phase of the sound waves reaching a point from all directions is irregular.
[0056] According to the principle of diffuse sound field, this application constructs a diffuse sound field model to represent the diffusion law of speech when the diffuse sound field conditions are met. The diffuse sound field model is used to calculate the weight vector of the beamformer, so that the characteristic estimation of noise is no longer calculated like the traditional enhancement algorithm, that is, beamforming can be completed without any prior information. In this way, the problems of poor robustness and low reliability of traditional enhancement algorithms are overcome, and target recognition can be well completed even in complex environments.
[0057] Figure 1 An exemplary system architecture of an embodiment of the speech enhancement method or speech enhancement device of the present application is shown. Figure 1 As shown, the system architecture includes a microphone array 101 , a network 102 and a terminal device 103 .
[0058] The microphone array 101 is a microphone array including sixteen microphones, which is used to collect voice signals and send the collected voice signals to the terminal device 103 through the network 102 .
[0059] The network 102 is used to provide a medium for a communication link between the microphone array 101 and the terminal device 103. The network 102 may include various connection types, such as wired and wireless communication links, and the like.
[0060] The terminal device 103 is used to receive the voice signal sent by the microphone array 101 and perform voice enhancement processing on the voice signal. The terminal device 103 can be hardware or software. When the terminal device 103 is hardware, it can be various computer devices, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, smart air conditioners, smart speakers, smart speakers, etc. When the terminal device 103 is software, it can be installed in the computer devices listed above.
[0061] It should be noted that the microphone array 101 and the terminal device 103 may be independent of each other, or the microphone array 101 may also be included in the terminal device 103, which is not specifically limited here.
[0062] It should be understood that Figure 1 The number of microphone arrays, networks, and terminal devices in the embodiment is only for illustration purposes. Any number of microphone arrays, networks, and terminal devices may be provided as required.
[0063] refer to Figure 2 , shows a flow chart of an embodiment of a speech enhancement method according to the present application, which specifically includes the following steps:
[0064] 201. Preprocess the speech signal containing the target speech collected by the microphone array to obtain time-frequency spectrum data of the speech signal.
[0065] In this embodiment, the execution subject of the voice enhancement method may be a terminal device (eg Figure 1 The terminal device can be connected to a microphone array (eg, Figure 1 The microphone array 101 is shown in the figure. The microphone array and the terminal device may be independent of each other, or the microphone array may be included in the terminal device, which is not specifically limited here.
[0066] The sixteen microphones in the microphone array send the collected voice signals to the terminal device. After receiving the voice signals, the terminal device preprocesses the voice signals and transforms the time domain signals into frequency domain signals.
[0067] In some optional implementations of this embodiment, since the speech signal collected by the microphone array belongs to the time domain signal, directly beamforming the time domain signal will greatly increase the computational difficulty. Based on this, the time domain signal is converted into a frequency domain signal for processing. The above preprocessing process can be as follows:
[0068] The speech signal containing the target speech collected by the microphone array is framed, windowed and Fourier transformed. Here, the frame length of the frame can be between 20 and 50 milliseconds. After framing, the framed signal needs to be windowed to reduce spectrum leakage. The window function can choose triangular window, Hanning window, Hamming window, Gaussian window, etc. After windowing, each frame signal is Fourier transformed to obtain the time-frequency spectrum data of the speech signal. Since framing, windowing and Fourier transform are currently widely studied and applied known technologies, they will not be repeated here.
[0069] 202. Based on a pre-constructed diffuse sound field model, the coordinates of each microphone element in the microphone array, and the arrival angle of the speech signal, determine the weight vector of the beamformer, and use the weight vector to perform beamforming on the time-frequency spectrum data; wherein the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field conditions are met.
[0070] In some optional implementations of this embodiment, the method for constructing the diffuse sound field model includes:
[0071] Determine the relative distance between each microphone array element according to the coordinates of all microphone array elements;
[0072] The diffuse sound field matrix corresponding to each frequency point is determined based on the relative distance between each microphone array element, the sound wave frequency and the sound speed of the speech signal;
[0073] The diffuse sound field matrices corresponding to all frequency points are combined to form a diffuse sound field model.
[0074] The microphone array is used as Figure 1 The microphone array 101 shown is used as an example for explanation. The coordinates of the sixteen microphones are (xi, yi, zi), i = 1, 2, 3, ..., 16. First, the relative distances between the microphone array elements are determined according to the coordinates of all the microphone array elements:
[0075]
[0076] Then, the diffuse sound field matrix corresponding to each frequency point is determined based on the relative distance between the microphone array elements, the sound wave frequency and sound speed of the speech signal:
[0077]
[0078] Where pi is the circumference of a circle, f is the sound wave frequency of the speech signal, and c is the sound speed of the speech signal;
[0079] Finally, the diffuse sound field matrices corresponding to all frequency points are combined to form a diffuse sound field model. Here, each frequency point corresponds to a 16×16 Gama, and the Gamas of all frequency points form a 16×16×(NFFT / 2+1) three-dimensional array GamaDum, where NFFT is the number of sampling points taken by Fourier transform.
[0080] After the diffuse sound field model is constructed, the weight vector of the beamformer can be determined based on the diffuse sound field model, the coordinates of each microphone element in the microphone array, and the arrival angle of the speech signal, and the weight vector can be used to perform beamforming on the time-frequency spectrum data.
[0081] In some optional implementations of this embodiment, the step of constructing a weight vector of a beamformer may include:
[0082] The diffuse sound field model is determined as the correlation matrix R N ;
[0083] The steering vector a is determined based on the sound wave frequency and sound speed of the speech signal, the coordinate matrix formed by the coordinates of each microphone array element, and the arrival angle of the speech signal. s ;
[0084] According to the correlation matrix R N , steering vector a s , the beamformer weight vector W is calculated using the preset calculation model opt ;
[0085] The calculation model is:
[0086] Here, the steering vector a s The calculation method is:
[0087]
[0088] tau = mics*ed;
[0089] Among them, i is the serial number of the microphone, pi is the pi, f is the sound wave frequency of the speech signal, c is the sound speed of the speech signal, mics is the coordinate matrix composed of the coordinates of each microphone array element, and ed is the direction vector of the angle of arrival of the speech signal.
[0090] It should be noted that the direction vector of the above-mentioned arrival angle is a three-dimensional vector obtained by the horizontal angle and the elevation angle of the voice signal relative to the microphone array. The calculation method of the direction vector of the arrival angle is a well-known technology and will not be repeated here.
[0091] The beamformer weight vector W is calculated opt Then, use W opt Beamforming is performed on the time-frequency spectrum data as follows:
[0092] Y=W opt H X;
[0093] Wherein, X is the time-frequency spectrum data of the speech signal, and Y is the expected data, that is, the data after beamforming.
[0094] 203. Filter the beamformed data using a post filter.
[0095] In some optional implementations of this embodiment, the post-filter may be any one of a minimum mean square error estimator (such as a Wiener filter), a log spectrum amplitude estimator, a short-time spectrum amplitude estimator, and a Zelinski filter. The research on the above-mentioned post-filters is relatively extensive, so the process of filtering using the post-filter is not repeated here.
[0096] 204. Perform transformation processing on the filtered data to obtain enhanced target speech.
[0097] Since the filtered data still belongs to the frequency domain signal, in order to obtain the target speech, the frequency domain signal needs to be converted to obtain the time domain signal. Optionally, the process of transforming the filtered data includes the following steps:
[0098] Perform inverse Fourier transform on the filtered data;
[0099] The data obtained after inverse Fourier transform is reconstructed using the stack-add method to obtain the enhanced target speech.
[0100] The method provided by the above-mentioned embodiment of the present application constructs a diffuse sound field model and designs the weight vector of the beamformer using data such as the diffuse sound field model. In this way, when recognizing the target speech, there is no need to calculate the characteristic estimation of the noise (the characteristic estimation of the noise is different in different scenarios), that is, the target speech recognition can be completed without relying on prior information, and the robustness and reliability are high.
[0101] Continue to see Figure 3 , which shows a flow chart of another embodiment of the speech enhancement method according to the present application, which specifically includes the following steps:
[0102] 301. Calculate correlation coefficients between microphone channels based on a diffuse sound field model.
[0103] 302. Calculate the post-filter coefficient of the Wiener filter by using the correlation coefficient between the microphone channels, the estimated value of the auto-power spectrum of the speech in the speech signal, and the estimated value of the auto-power spectrum of the noise.
[0104] 303. Perform Wiener filtering on the beamformed data according to the post-filter coefficients.
[0105] Here, the correlation coefficient between each microphone channel is calculated by the following formula:
[0106]
[0107] Here, W McCowan is the coherence matrix, N is the number of microphone channels, is the correlation coefficient between the ith microphone channel and the jth microphone channel, for The estimated value of .
[0108] After calculating the correlation coefficients between the microphone channels, the post-filter coefficients of the Wiener filter can be calculated using the correlation coefficients between the microphone channels, the estimated values of the auto-power spectrum of the speech in the speech signal, and the estimated values of the auto-power spectrum of the noise. The specific calculation formula is as follows:
[0109]
[0110] Among them, h p is the post-filter coefficient, is the estimated value of the speech autopower spectrum, is an estimate of the autopower spectrum of the noise.
[0111] here, Based on the time-frequency spectrum data, the correlation coefficient is calculated according to the given calculation method. The specific calculation method is as follows:
[0112]
[0113]
[0114]
[0115]
[0116]
[0117] in, It is the time-frequency spectrum data corresponding to each microphone channel.
[0118] According to the above formula, we can get:
[0119]
[0120]
[0121] in, They are The estimated value of .
[0122] Calculate the post-filter coefficient h p After that, use h p Perform Wiener filtering on the beamformed data.
[0123] In this embodiment, a diffuse sound field model is used to obtain a correlation coefficient, and the filter coefficient of the Wiener filter is designed by using data such as the correlation coefficient, so that the target speech can be further enhanced. When multiple people are speaking, interference from other voices can be effectively suppressed, and interference from environmental noise can also be effectively suppressed.
[0124] Continue to see Figure 4 , which shows a flow chart of another embodiment of the speech enhancement method according to the present application, which specifically includes the following steps:
[0125] 401. Before performing Wiener filtering on beamformed data according to the post-filter coefficients, smoothing the post-filter coefficients.
[0126] Here, smoothing can be performed as follows:
[0127] h p _new=0.8*h p _old+0.2*h p _now;
[0128] Among them, h p _new is the post-filter coefficient after smoothing, h p _old is the post-filter coefficient of the previous frame, h p _now is the post-filter coefficient of the current frame.
[0129] 402. Perform median filtering on the post-filter coefficients obtained after the smoothing process.
[0130] Among them, median filtering means setting the gray value of each pixel point to the median of the gray values of all pixels in a certain area window of the point. Median filtering can select multiple points for filtering, such as 5 points, 10 points, etc., which can be set according to actual needs.
[0131] Further references Figure 5 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a speech enhancement device, which is similar to Figure 2 Corresponding to the method embodiment shown, the apparatus can be specifically applied to various computer devices.
[0132] like Figure 5 As shown, the speech enhancement device of this embodiment includes:
[0133] The data processing unit 501 is used to pre-process the speech signal containing the target speech collected by the microphone array to obtain the time-frequency spectrum data of the speech signal;
[0134] The beamforming unit 502 is used to determine the weight vector of the beamformer based on the pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal, and use the weight vector to beamform the time-frequency spectrum data; wherein the diffuse sound field model is used to represent the diffusion law of the speech when the diffuse sound field condition is met;
[0135] A post-filtering unit 503, configured to filter the beamformed data using a post-filter;
[0136] The data processing unit 501 is also used to perform transformation processing on the filtered data to obtain enhanced target speech.
[0137] In some optional implementations of this embodiment, the post-filter is any one of a minimum mean square error estimator, a logarithmic spectrum amplitude estimator, a short-time spectrum amplitude estimator and a Zelinski filter.
[0138] In some optional implementations of this embodiment, the post filter is a Wiener filter, and the post filtering unit 503 is specifically used for:
[0139] Calculate the correlation coefficient between each microphone channel based on the diffuse sound field model;
[0140] The post-filter coefficient of the Wiener filter is calculated by using the correlation coefficient between the microphone channels, the estimated value of the auto-power spectrum of the speech in the speech signal, and the estimated value of the auto-power spectrum of the noise;
[0141] Performing Wiener filtering on the beamformed data according to the post-filter coefficients;
[0142] The estimated value of the auto-power spectrum of speech and the estimated value of the auto-power spectrum of noise in the speech signal are determined by calculating the time-frequency spectrum data and the correlation coefficient according to a given calculation method.
[0143] In some optional implementations of this embodiment, the above device further includes:
[0144] The smoothing and filtering unit 504 is used to perform smoothing and median filtering on the post-filter coefficients before performing Wiener filtering on the beamformed data according to the post-filter coefficients.
[0145] In some optional implementations of this embodiment, the beamforming unit 502 is specifically configured to:
[0146] The diffuse sound field model is determined as the correlation matrix R N ;
[0147] The steering vector a is determined based on the sound wave frequency and sound speed of the speech signal, the coordinate matrix formed by the coordinates of each microphone array element, and the arrival angle of the speech signal. s ;
[0148] According to the correlation matrix R N , steering vector a s , the beamformer weight vector W is calculated using the preset calculation model opt ;
[0149] The calculation model is:
[0150] In some optional implementations of this embodiment, the above device further includes:
[0151] A construction unit 505, configured to determine the relative distances between the microphone array elements according to the coordinates of all the microphone array elements;
[0152] The diffuse sound field matrix corresponding to each frequency point is determined based on the relative distance between each microphone array element, the sound wave frequency and the sound speed of the speech signal;
[0153] The diffuse sound field matrices corresponding to all frequency points are combined to form a diffuse sound field model.
[0154] In some optional implementations of this embodiment, the data processing unit 501 is used to:
[0155] The speech signal containing the target speech collected by the microphone array is framed, windowed and Fourier transformed.
[0156] In some optional implementations of this embodiment, the data processing unit 501 is further used to: perform inverse Fourier transform on the filtered data;
[0157] The data obtained after inverse Fourier transform is reconstructed using the stack-add method to obtain the enhanced target speech.
[0158] Reference below Figure 6 , which shows a computer device (eg, Figure 1A schematic diagram of the structure of the terminal device 103 shown in FIG. Figure 6 The computer device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0159] in, Figure 6 The computer device shown includes a processor, a memory and a network interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a speech enhancement method is implemented.
[0160] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0161] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0162] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A speech enhancement method, characterized in that: The method comprises: Preprocessing the speech signal containing the target speech collected by the microphone array to obtain time-frequency spectrum data of the speech signal; Based on a pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal, a weight vector of a beamformer is determined, and beamforming is performed on the time-frequency spectrum data using the weight vector; wherein the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field condition is met; Filtering the beamformed data using a post filter; Transform the filtered data to obtain enhanced target speech; The step of determining the weight vector of the beamformer based on the pre-built diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal specifically includes: The diffuse sound field model is determined as the correlation matrix R N ; The steering vector a is determined based on the sound wave frequency and sound speed of the speech signal, the coordinate matrix formed by the coordinates of each microphone array element, and the arrival angle of the speech signal. s ; According to the correlation matrix R N , steering vector a s , the beamformer weight vector W is calculated using the preset calculation model opt ; Wherein, the calculation model is: The steering vector a s The calculation method is tau = mics*ed, where i is the serial number of the microphone, pi is the circumference of a circle, f is the sound wave frequency of the speech signal, c is the sound velocity of the speech signal, mics is the coordinate matrix composed of the coordinates of each microphone array element, and ed is the direction vector of the angle of arrival of the speech signal; The method for constructing the sound field model comprises: Determine the relative distance between each microphone array element according to the coordinates of all microphone array elements; Based on the relative distance between each microphone array element, the sound wave frequency and sound speed of the speech signal, the diffuse sound field matrix corresponding to each frequency point is determined. The formula is: Among them, pi is pi, f is the sound wave frequency of the speech signal, c is the sound speed of the speech signal, and dij is the relative distance between array elements; The diffuse sound field matrices corresponding to all frequency points are combined to form the diffuse sound field model.
2. The method according to claim 1, characterized in that The post-filter is any one of a minimum mean square error estimator, a logarithmic spectrum amplitude estimator, a short-time spectrum amplitude estimator and a Zelinski filter.
3. The method according to claim 1, characterized in that The post filter is a Wiener filter, and the filtering of the beamformed data by the post filter specifically includes: Calculating correlation coefficients between microphone channels based on the diffuse sound field model; The post-filter coefficient of the Wiener filter is calculated by using the correlation coefficient between the microphone channels, the estimated value of the auto-power spectrum of the speech in the speech signal, and the estimated value of the auto-power spectrum of the noise; Performing Wiener filtering on the beamformed data according to the post-filter coefficients; The estimated value of the auto-power spectrum of speech and the estimated value of the auto-power spectrum of noise in the speech signal are determined based on the time-frequency spectrum data and the correlation coefficient calculated according to a given calculation method.
4. The method according to claim 3, characterized in that Before performing Wiener filtering on the beamformed data according to the post-filter coefficient, the method further includes: The post-filter coefficients are smoothed and median filtered.
5. The method according to claim 1, characterized in that The preprocessing of the speech signal containing the target speech collected by the microphone array specifically includes: The speech signal containing the target speech collected by the microphone array is framed, windowed and Fourier transformed.
6. The method according to claim 1, characterized in that The filtered data is transformed to obtain the enhanced target speech, which specifically includes: Perform inverse Fourier transform on the filtered data; The data obtained after inverse Fourier transform is reconstructed using the stack-add method to obtain the enhanced target speech.
7. A speech enhancement device, characterized in that: The device is used to implement the speech enhancement method according to any one of claims 1 to 6, and the device comprises: A data processing unit, used to pre-process the speech signal containing the target speech collected by the microphone array to obtain time-frequency spectrum data of the speech signal; A beamforming unit, configured to determine a weight vector of a beamformer based on a pre-constructed diffuse sound field model, the coordinates of each microphone array element in the microphone array, and the arrival angle of the speech signal, and to perform beamforming on the time-frequency spectrum data using the weight vector; wherein the diffuse sound field model is used to represent the diffusion law of speech when the diffuse sound field condition is met; A post-filtering unit, used for filtering the beamformed data using a post-filter; The data processing unit is also used to perform transformation processing on the filtered data to obtain enhanced target speech.
8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
