Sound source localization and enhancement method for binaural hearing aid

Abstract

The invention relates to the technical field of hearing aid sound source localization, in particular to a sound source localization and enhancement method for a binaural hearing aid, and the method comprises the steps: determining a sound source angle in any time period according to the relevance between received signals of different microphones in any time period in combination with random maximum likelihood estimation; determining the existence probability of each sound source at the sound source angle in any time period by analyzing the distribution condition of each sound source and combining a Gaussian mixture model; based on the existence probability of each sound source under the sound source angle of any time period, determining the dynamic existence probability of each sound source under the sound source angle of any time period; and enhancing the sound sources based on the dynamic existence probability of each sound source at the sound source angle of the current time period. The invention aims to improve the sound source positioning precision and enhance the intensity of the sound source signal received by the hearing aid.

Description

Technical Field

[0001] The present application relates to the technical field of sound source localization for hearing aids, and in particular to a sound source localization and enhancement method for binaural hearing aids. Background Art

[0002] With the aging of society and the increase in the number of hearing-impaired people, the demand for hearing aids is growing. Hearing aids (HAs) are assistive devices used to help hearing-impaired people improve their hearing experience by amplifying sound. Most modern hearing aids use multi-microphone arrays and advanced signal processing algorithms to improve speech clarity and suppress background noise. These hearing aids use beamforming technology to enhance sounds from specific directions through spatial filtering while suppressing sounds from other directions.

[0003] In the process of improving various beamforming algorithms and applying them to binaural hearing aids in the prior art, the accuracy and robustness of sound source localization are insufficient when facing complex acoustic environments, such as dealing with multiple sound sources, strong interference and reverberation. Algorithms such as SML and AML are used to reduce the computational complexity of the maximum likelihood estimation method, but they are not specifically involved in hearing aid application scenarios, and the head-related transfer function is not taken into account. When applied to binaural hearing aids, the accuracy of sound source localization is reduced due to the influence of head rotation, and at the same time the strength of the sound source signal received by the hearing aid is reduced. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a sound source localization and enhancement method for binaural hearing aids to solve the existing problems.

[0005] The present invention discloses a method for localizing and enhancing a sound source for binaural hearing aids using the following technical solutions: An embodiment of the present application provides a method for sound source localization and enhancement for binaural hearing aids, the method comprising the following steps: There are multiple sound sources in the environment, and the receiving signals of each microphone within a preset time length are obtained, and the preset time length is divided into multiple time periods; According to the correlation between the received signals of different microphones at any time period, combined with random maximum likelihood estimation, the sound source angle at any time period is determined; At the sound source angle in any period of time, by analyzing the distribution of each sound source and combining it with the Gaussian mixture model, the existence probability of each sound source at the sound source angle in any period of time is determined; Based on the existence probability of each sound source at the sound source angle in any period of time, determine the dynamic existence probability of each sound source at the sound source angle in any period of time; Based on the received signals of each microphone in the current time period, the beam of the current time period is obtained; based on the dynamic existence probability of each sound source at the sound source angle of the current time period, the iteration step of each sound source at the sound source angle of the current time period is determined, and the beam of the current time period and the received signals of each microphone are combined to determine the adaptive noise elimination coefficient of each sound source in the next time period to enhance the sound source.

[0006] Preferably, the method for determining the sound source angle in any time period is: The set of received signals of all microphones in any period of time is taken as the microphone signal, and the covariance matrix of the microphone signal is obtained as the input of the random maximum likelihood estimation algorithm. The random maximum likelihood estimation algorithm is iterated to output the sound source angle in any period of time.

[0007] Preferably, the method for determining the existence probability of each sound source at the sound source angle in any time period is: The expression of the existence probability of sound source q under the sound source angle of time period b is for: ; In the formula, represents the prior probability of the qth Gaussian component in the Gaussian mixture model, where one Gaussian component represents one sound source; represents the qth Gaussian component; represents the angle of the sound source in time period b; , Respectively represent the means of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model; , They represent the variance of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model respectively; represents the Gaussian distribution function; Q represents the number of all Gaussian components.

[0008] Preferably, the prior probability, mean and variance of the Gaussian component are estimated by an expectation-maximization algorithm.

[0009] Preferably, the method for determining the dynamic existence probability of each sound source is: The existence probability of each sound source under the sound source angle in any period of time is used as the input of the speech separation SPP algorithm, and the dynamic existence probability of each sound source under the sound source angle in any period of time is output.

[0010] Preferably, the expression of the iteration step length of each sound source under the sound source angle of the current time period is: ; In the formula, Indicates the iteration step of the sound source q at the sound source angle of the current period; Indicates the probability of existence of sound source q under the sound source angle of the current period; Indicates a preset value.

[0011] Preferably, the process of acquiring the beam in the current time period is: The microphone signal in the current time period is used as the input of the minimum variance distortion-free response algorithm, and the beam in the current time period is output.

[0012] Preferably, the step of determining the adaptive noise elimination coefficient of each sound source in the next time period includes: Adaptive noise elimination coefficient of sound source q in the next period The expression is: ; In the formula, represents the adaptive noise elimination coefficient of the sound source q in the current period, where: The initial value of is 0; Indicates the beam in the current period; Represents the adaptive blocking matrix coefficient of the sound source q in the next period; Indicates the microphone signal in the current period; Indicates taking the absolute value.

[0013] Preferably, the adaptive blocking matrix coefficients of the sound source q in the next time period are the output results obtained by taking the microphone signal in the next time period and the dynamic existence probability of the sound source q as the input of the generalized sidelobe canceller.

[0014] Preferably, the enhancing the sound source comprises: The microphone signal of the next time period is used as the input of the minimum distortion-free variance, and a preliminary enhanced signal is output, and the enhanced signal is used as the input of the generalized sidelobe cancellation technology, wherein the adaptive noise elimination coefficients of each sound source in the next time period are used as the adaptive filter coefficients in the adaptive filter of the generalized sidelobe canceller, and the final enhanced signal is output.

[0015] This application has at least the following beneficial effects: This application optimizes the identification and tracking of sound sources by fusing beamforming technology and statistical models; and further improves the accuracy of sound source localization by introducing multi-channel speech presence probability (SPP) estimation to optimize the Gaussian mixture model (GMM). Furthermore, the performance of noise cancellation is enhanced by adapting the blocking matrix and noise canceller of the generalized sidelobe canceller CGC; this application can maintain high performance even when the user's head turns by considering the head correlation function; the advantage of this application is that it can process audio signals in real time and adapt to different users without prior training, making it very real-time and universal. This application can provide an improved auditory experience under a variety of noise conditions, improve the accuracy of sound source localization, and enhance the strength of the sound source signal received by the hearing aid. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flowchart of a method for sound source localization and enhancement for binaural hearing aids provided in accordance with an embodiment of the present application; Figure 2 A flow chart of sound source localization provided for one embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the sound source localization and enhancement method for binaural hearing aids proposed in the present application, its specific implementation method, structure, features and effects, in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0019] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0020] A specific solution of a sound source localization and enhancement method for binaural hearing aids provided by the present application is described in detail below with reference to the accompanying drawings.

[0021] An embodiment of the present application provides a method for localizing and enhancing a sound source for a binaural hearing aid. Specifically, the following method for localizing and enhancing a sound source for a binaural hearing aid is provided. Figure 1 , the method comprises the following steps: Step S1: Obtain the received signals of each microphone within a preset time length, and divide the preset time length into multiple time periods. The binaural hearing aid described in this embodiment uses a microphone array to capture sounds from different directions, which includes at least four microphones. The microphone array can be any of the following: a one-dimensional linear array, a two-dimensional surface array, and a three-dimensional stereo array. Obtain the received signals of each microphone within a preset time length, and divide the preset time length into multiple time periods.

[0022] Preferably, two microphones are installed in the unilateral hearing aid, the distance between them is 15.32 mm, and the installation position is at the ear. , where n represents the sampling time, T represents the transpose, and the frequency domain is expressed as , where f represents the frequency point. The sampling frequency of the microphone is between 8kHz and 48kHz, and the collected signal is processed in segments, that is, frame processing, and the length of each frame is 20ms.

[0023] In addition, it should be noted that the value of the preset time length is manually set. In this embodiment, the value of the preset time length is 1s. The implementer can also set it by himself according to the specific situation. This embodiment does not impose any special restrictions.

[0024] Step S2: Determine the sound source angle at any time period based on the correlation between the received signals of different microphones at any time period and in combination with random maximum likelihood estimation.

[0025] Traditional generalized sidelobe cancellers GSC usually process the signals received by the microphone array based on a preset directional diagram or algorithm, such as minimum variance distortion-free response MVDR. Its input is the original audio signal captured by the microphone array containing multiple sound sources and noise. By performing a weighted summation operation on these signals, a main beam is formed in the desired direction, so that the target sound source direction has a relatively high gain, but it may still contain certain interference and noise. The main beam is the direction of the sound source, but its sound source positioning mostly depends on a fixed beam direction and noise. The main beam is the direction of the sound source, but its sound source mostly depends on a fixed beam direction and simple signal difference judgment. In a complex environment with multiple sound sources, it is difficult to accurately identify the direction of each sound source, and positioning deviations are prone to occur, affecting the subsequent signal processing effect.

[0026] In this embodiment, the random maximum likelihood estimation SML function is used to preliminarily estimate the direction of the sound source. Using the random maximum likelihood method, taking into account the head-related transfer function and the position noise condition, the direction of the sound source is estimated by constructing a log likelihood function and maximizing it, which can more accurately locate the sound source in a complex acoustic environment, provide accurate sound source location information for subsequent signals, and improve the positioning accuracy and stability of the entire system.

[0027] The traditional stochastic maximum likelihood estimation SML is a logarithmic function of the probability density function of the multivariate Gaussian distribution. In this embodiment, the log-likelihood function is constructed based on the spatial covariance matrix of the signal and the spatial covariance matrix of the noise. From the perspective of information theory, the log-likelihood function is a logarithmic form that describes the probability of the observed data under given model parameters. For the signal received by the microphone array, we hope to find a parameter (sound source direction) so that the probability of the observed data X(n) occurring under this parameter is maximized. First, the power of the clean signal and the noise signal is estimated based on the sound signal received by the microphone, and then these estimates are used to optimize the log-likelihood function to extract information related to the direction of the sound source.

[0028] Therefore, according to the correlation between the received signals of different microphones in any period of time, combined with the random maximum likelihood estimation, the sound source angle in any period of time is determined, specifically: The set of received signals of all microphones in any period of time is taken as the microphone signal, and the covariance matrix of the microphone signal is obtained as the input of the random maximum likelihood estimation algorithm. The random maximum likelihood estimation algorithm is iterated to output the sound source angle in any period of time. In order to explain the principle of obtaining the sound source angle more specifically, the specific principle derivation process of the sound source angle is as follows: The microphone signal includes the received sound source signal and the noise signal, where the received sound source is recorded as the target signal. It is assumed that the target signal is an unknown deterministic signal and the noise signal follows a zero-mean complex Gaussian distribution. First, the probability density function of the microphone received signal is calculated through the covariance matrix, and the logarithm is taken to obtain the logarithmic likelihood function. Then, the data of the B frame is integrated, and the logarithmic likelihood function for a single sound source signal is: (1) Wherein, B represents the number of all time periods within the preset time length; M represents the number of microphones; Tr{} represents the trace of the matrix; Represents the direction of the sound source and is the parameter to be estimated. Its optimal value is determined by maximizing the log-likelihood function.

[0029] Represents the covariance matrix of the microphone signal, and the calculation formula is: (2) In the formula, Indicates the direction of the sound source Head-related transfer functions (HRTFs) associated with frequency k, taking into account the effects of head movement; Represents the power spectrum value of the target signal in the microphone signal at frequency k in the frequency domain; Represents the power spectrum of the noise signal in the microphone signal; Represents the covariance matrix of the noise signal; express The conjugate transpose of .

[0030] Among them, the process of obtaining the covariance matrix of the noise signal of the zero-mean complex Gaussian distribution is a well-known technology, and the head-related transfer function is a well-known technology, and the specific principles thereof are not repeated here.

[0031] The head-related transfer function is a function that describes the frequency response changes caused by the physiological structures of the head, auricle, trunk, etc. when the sound propagates from the sound source to the ears. It plays a vital role in the field of acoustic signal processing. When the sound propagates in space and reaches the ears, the head will block, reflect and diffract the sound, and the unique shape of the auricle will further modulate the sound signal.

[0032] These factors cause the amplitude and phase of the sound to change at different frequencies, and the head-related transfer function HRTFs can accurately quantify these changes. In this embodiment, by considering the head-related transfer function and multiple links of the signal processing, the system can take into account the impact of head-related factors on sound propagation, thereby more accurately locating the sound source and processing the signal. For example, in a complex acoustic environment, the sound emitted by the sound source in different directions will be modulated by the head-related transfer function, and the signal received at the two ears will show different characteristics. By analyzing these characteristic differences, the system can better distinguish the direction and distance of the sound source, improve the performance of the hearing aid in multiple sound sources and complex noise environments, and provide users with more accurate sound enhancement and clearer hearing experience.

[0033] Covariance matrix of microphone signal It consists of two parts, one of which is the head-related transfer function Covariance of the modulated target signal , and the other part is the noise covariance The head-related transfer function takes into account the influence of factors such as head occlusion and reflection on the sound during the process of sound propagating from the sound source to the microphone. It modulates the signal source at different frequencies and directions. The target signal power spectrum reflects the energy distribution of the target signal at different frequencies, the noise signal power spectrum represents the average power of the noise at each frequency, and the noise correlation matrix describes the correlation of noise between different microphones.

[0034] Among them, the power spectrum of the target signal and the noise signal is estimated as follows: (3) (4) In the power spectrum estimation formula (3) of the target signal, is the covariance matrix of the microphone signal, which contains the mixed information of the target signal and the noise signal; It is the head-related transfer function, which represents the linear transformation relationship of the sound from the sound source to the microphone. This transformation reflects the characteristic changes of the sound at different frequencies and directions due to the influence of the head. It is equivalent to preliminarily removing the influence of noise from the mixed covariance information, which is a feature screening operation; then and The matrix multiplication operation is to use the combination of the conjugate transpose of the head-related transfer function and the inverse of the noise correlation matrix to further linearly transform the preliminarily processed signal to highlight the characteristics of the target signal, where: represents the inverse matrix of the covariance matrix of the noise signal; finally, normalization is performed through the operation of the denominator, so that the obtained It can accurately represent the power spectrum characteristics of the target signal at different frequencies, that is, it achieves the purpose of extracting the power spectrum of the target signal from the mixed signal.

[0035] In the power spectrum estimation formula (4) of the noise signal, Tr{} represents the trace of the matrix; represents the identity matrix; It involves inverting the matrix related to the head-related transfer function and the noise correlation matrix in order to adjust the signal space so as to extract the characteristics of the noise signal later; The matrix multiplication operation and matrix trace operation are to comprehensively process the adjusted signal and extract the power spectrum information of the noise signal from the overall signal and matrix relationship. The matrix trace operation is equivalent to summing or extracting certain features of the matrix after a series of transformations, and finally obtaining the power spectrum of the noise signal, achieving the goal of accurately estimating the noise power spectrum from a complex signal and matrix combination.

[0036] First, the initial covariance matrix of the signal is calculated using the traditional method, and the target signal power spectrum estimate is obtained according to formula (3), and the noise signal power spectrum estimate is obtained according to formula (4). The covariance is then updated using the covariance matrix formula (2), and then the covariance matrix, the target signal power spectrum, and the noise signal power spectrum estimate are gradually adjusted through iteration, so that they are closer and closer to the true values.

[0037] Substituting the final estimated noise power and signal power into the log-likelihood function formula (1), we get the objective function (5): (5) in, and are two auxiliary matrices used to project and process the target signal and noise. The calculation is obtained by formula (6) and formula (7), specifically: (6) (7) The construction process is actually a recombining and adjusting operation of the signal and noise after being modulated by HRTFs. And find the inverse, and then and The purpose of performing the multiplication operation is to highlight the target signal in the construction and subsequent calculation of the log-likelihood function, so that in a complex acoustic environment, the information related to the target sound source can be more effectively separated from the mixed signal.

[0038] And for , it is The complementary space matrix of is used to shield the target signal. In the calculation of the log-likelihood function and the entire signal processing process, when When used to extract the target signal, Other parts that are not related to the target signal (mainly noise and interference signals) can be processed and isolated. The two work together to make the entire signal processing process more accurate and efficient, thereby improving the accuracy of sound source positioning and signal enhancement.

[0039] Finally, the sound source angle is obtained by maximizing the objective function: (8) In the formula, represents the sound source angle in time period b; b represents time period b, It represents the number of all frequency points in the frequency domain in time period b; argmax() represents the maximum value index function.

[0040] Compared with the traditional stochastic maximum likelihood estimation (SML) method that directly performs sound source localization based on the probability density function of a simple signal model, this embodiment not only takes into account the calculation of the log-likelihood function of the covariance matrix of the target signal and the noise signal, but also takes into account the modulation of the target signal by the head-related transfer function, so as to better adapt to complex acoustic environments and improve the accuracy and robustness of sound source localization.

[0041] Among them, the stochastic maximum likelihood estimation SML method is a well-known technology, and its specific principle explanation is not repeated here.

[0042] At this point, the sound source angle is obtained.

[0043] Step S3: Under the sound source angle of any time period, by analyzing the distribution of each sound source and combining it with the Gaussian mixture model, the existence probability of each sound source under the sound source angle of any time period is determined.

[0044] Stochastic maximum likelihood estimation SML provides preliminary directional information of the sound source. However, in the real acoustic environment, there are multiple sound sources and the background noise is complex and changeable. The number, position and intensity of the sound sources, in order to improve the accuracy of positioning, need to perform statistics on the estimated narrowband angle. In this embodiment, the Gaussian mixture model GMM is used to calculate the existence probability of the target signal.

[0045] The Gaussian mixture model GMM can describe this complex probability distribution through a combination of multiple Gaussian components. Each Gaussian component can represent a potential sound source or noise source, and its mean and variance can capture the characteristics of the sound source or noise in spatial frequency. Moreover, the Gaussian mixture model GMM can adapt to such changes by adjusting the parameters of the Gaussian components (such as prior probability, mean and variance). When a new sound source appears or an existing sound source disappears, the Gaussian mixture model GMM can re-estimate the weights and parameters of each Gaussian component to reflect changes in the acoustic environment. This adaptive capability is crucial for devices such as hearing aids, because it needs to always maintain good performance in different usage scenarios and accurately identify and track the target sound source.

[0046] Assuming that there are Q sound sources, the probability of each sound source existing at the sound source angle in any period of time is determined as follows: The expression of the existence probability of sound source q under the sound source angle of time period b is for: (9) In the formula, represents the prior probability of the qth Gaussian component in the Gaussian mixture model, where one Gaussian component represents one sound source; represents the qth Gaussian component; represents the angle of the sound source in time period b; , They represent the expected positions of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model respectively; , They represent the position variance of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model respectively; represents the Gaussian distribution function; Q represents the number of all Gaussian components.

[0047] The prior probability, mean and variance of the Gaussian component are estimated by the expectation maximization algorithm.

[0048] Based on the general understanding of the distribution of sound sources in different acoustic environments or empirical estimates obtained through statistical analysis of a large amount of experimental data. When calculating the probability of the existence of a sound source, it is used as the initial weight of each Gaussian component, combined with the Gaussian distribution function, to affect the final probability calculation result. It reflects the relative importance of each potential sound source or noise source in the model when there is no specific observation data. As the system receives and processes new audio signals, its value will be dynamically adjusted according to the actual data in the subsequent iteration of the expectation maximization (EM) algorithm to more accurately reflect the actual situation of each sound source in the current acoustic environment.

[0049] is the expected position of the qth sound source, represents the position variance of the qth sound source, which is also set in the initial stage based on the understanding of common acoustic signal characteristics. In terms of speech signals, a rough initial mean and variance are determined based on factors such as the frequency range of speech, its propagation characteristics in space, and its performance in different environments. As the system runs, these two parameters are continuously updated by analyzing and processing the signals received by the microphone array and performing statistical analysis on continuous time frame data using the EM algorithm. In each iteration, their values ​​are adjusted based on the degree of match between the current data point and each Gaussian component (calculated by the likelihood function) to more accurately reflect the characteristic changes of the actual sound source in space and frequency.

[0050] ( ) represents the Gaussian distribution function, Represents the estimated value of the sound source direction. The entire formula calculates the ratio of the probability contribution of a specific sound source to the sum of the probability contributions of all sound sources to obtain the probability of the existence of the qth sound source given the current estimated sound source direction. This calculation method can comprehensively consider multiple potential sound sources or noise sources in a complex acoustic environment, and accurately evaluate the possibility of each sound source existing in each time-frequency frame based on their respective prior probabilities, position characteristics, and current sound source direction estimates, thereby providing a key basis for subsequent signal processing decisions (such as beamforming, noise suppression, etc.), and helping to improve the performance of hearing aids in sound source localization and signal enhancement in complex acoustic scenarios.

[0051] In this embodiment, the expectation maximization EM algorithm is used to estimate the mean (representing the direction of the sound source), variance (representing the uncertainty of the direction) and mixing coefficient (prior probability) of each Gaussian component. The EM algorithm performs statistics on a continuous time frame, first starting with an initial guess, then calculating the expected value of the hidden variable in the expectation (E) step, and then re-estimating the model parameters in the maximization (M) step to maximize the likelihood function of the observed data. This iterative process is repeated until it converges to a set of optimal parameters, so that the model can accurately reflect the statistical characteristics of the sound source and noise.

[0052] The expectation maximization EM algorithm can adapt to changes in the number of sound sources, automatically estimate the number of active sound sources in the environment, and increase the flexibility and adaptability of the system. Through these optimized parameters, the beamformer can more effectively separate the target sound source, improve speech clarity and reduce background noise, significantly improving the hearing experience of hearing aid users.

[0053] Traditional sound source localization is mainly based on the arrival time difference method, which calculates the sound source position by measuring the time difference between the sound signal reaching different microphones. Its principle is based on simple geometric relationships and the assumption of signal propagation speed. The beamforming method is to perform weighted summation on the signals received by the microphone array so that the signal in a specific direction is enhanced, thereby determining the direction of the sound source. This mainly relies on the principle of spatial filtering in signal processing.

[0054] Traditional methods can achieve good results in relatively simple environments with less noise. However, since they do not fully consider the complex probability distribution of acoustic signals in actual environments and the uncertainty of noise, their positioning accuracy and reliability will be significantly affected in complex environments, such as those with multipath propagation, reverberation, and non-stationary noise.

[0055] However, the actual acoustic environment is often dynamically changing, and the number, location, intensity, and characteristics of the sound sources may change significantly over time. Traditional sound source localization methods are usually based on static model assumptions and fixed parameter settings, which are difficult to quickly and effectively adapt to such a dynamically changing environment, resulting in a sharp drop in positioning accuracy and reliability when the environment changes.

[0056] Therefore, this scheme introduces the existence probability of the sound source and combines it with the improved random maximum likelihood estimation SML to preliminarily determine the direction of the sound source. By constructing the log-likelihood function and maximizing it, the direction of the sound source can be estimated based on complex factors such as head-related transfer functions (HRTFs) and unknown noise conditions. However, in actual complex acoustic environments, relying solely on the random maximum likelihood estimation SML method may not be able to fully and accurately describe the probability distribution of the signal, thereby affecting the positioning accuracy. The Gaussian mixture model GMM describes complex probability distributions through the combination of multiple Gaussian components, and can more carefully characterize the characteristics of acoustic signals at different frequencies and spaces. Combining the Gaussian mixture model GMM with the random maximum likelihood estimation SML, the Gaussian mixture model GMM can provide a more accurate signal probability distribution model for the random maximum likelihood estimation SML, so that the random maximum likelihood estimation SML can better consider the uncertainty and complex probability distribution of the signal when calculating the log-likelihood function and estimating the direction of the sound source, thereby significantly improving the accuracy of sound source localization. In harsh environments such as multiple sound sources, strong noise and complex reflections, the location of the sound source can still be determined more accurately.

[0057] Among them, Gaussian mixture model GMM, Gaussian distribution function and expectation-maximization EM algorithm are all well-known technologies, and their specific principles are not repeated here.

[0058] At this point, the probability of the sound source existing is obtained.

[0059] Step S4: Based on the existence probability of each sound source at the sound source angle in any time period, determine the dynamic existence probability of each sound source at the sound source angle in any time period.

[0060] Since in the Gaussian mixture model GMM, the position of each sound source is represented by a Gaussian component, its prior probability and variance are usually fixed or set based on experience. This method is difficult to accurately capture the probability of the existence of sound sources in the actual environment in a complex and changeable acoustic environment, such as the user's head movement, the sudden appearance or disappearance of surrounding sound sources, changes in noise intensity and frequency characteristics, etc., especially when the noise conditions change or the number of sound sources is uncertain. SPP is used for improvement in this embodiment, specifically: the existence probability of each sound source at the sound source angle in any time period is used as the input of the speech separation SPP algorithm, and the dynamic existence probability of each sound source at the sound source angle in any time period is output. For ease of understanding, a more specific derivation process is given below, specifically: The speech separation SPP algorithm estimates that an additional Gaussian component is introduced into the Gaussian mixture model GMM to represent the noise. The mean of this noise component is set to the direction of the noise, and the variance is set to the power of the noise. In this way, GMM can model not only the sound source, but also the noise environment. For each sound source q, two assumptions are defined: Assume that the sound source exists , assuming that the sound source does not exist and only noise exists Using this assumption, the expression of the dynamic existence probability of each sound source at the sound source angle in any period of time is: (10) In the formula, represents the posterior probability to be calculated, that is, the probability that the qth sound source exists when the observed data X is known; It represents the probability of the observed data X appearing under the condition that the sound source q exists. It measures the rationality of the observed data X under the assumption that the sound source exists. Represents the prior probability of the existence of the qth sound source, that is, the probability estimate of the existence of the qth sound source before observing the data X. This probability is usually obtained based on historical data or domain knowledge; represents the sum of the products of the likelihood function and the prior probability of the existence of all sound sources. Here, all possible sound sources are considered (there are Q sound sources in total) and their probability of existence is weighted and summed; It represents the probability of the observed data X appearing when only noise exists. It measures the rationality of the observed data X when there is no sound source (only noise). It represents the prior probability of no speech, that is, the probability of only noise in the absence of any sound source.

[0061] in, It represents the prior probability of the existence of the qth sound source. Its acquisition method is a well-known technology, and the specific acquisition process will not be repeated here. It serves as an important weighting factor when calculating the probability of the sound source. It reflects the possibility of the existence of the sound source q based on past experience or statistical analysis of a specific environment before there is specific observation data. In the subsequent construction of the likelihood ratio function, it is combined with the probability part calculated based on the observation data to jointly determine the final existence probability of the sound source q under the given signal X. It is usually obtained by analyzing and counting a large amount of audio data. In the application scenario of hearing aids, audio samples may be collected in different environments, and information such as the frequency and position of the sound sources in them are statistically analyzed to determine the prior probability of different sound sources.

[0062] Represents the prior probability of no speech. Its acquisition process is a well-known technology and the specific principle will not be repeated here. It corresponds to the prior probability of the existence of a sound source. It represents the prior probability in the absence of a speech signal (i.e., only noise). In the likelihood ratio function, it is used as part of the denominator to measure the relative likelihood of the observed data under the assumption of no speech and compare it with the situation under the assumption of the existence of a sound source. The ratio and other related calculations can highlight the difference between the presence and absence of a sound source, thereby more accurately determining whether a sound source exists.

[0063] Define the generalized likelihood ratio, expressed as: (11) The likelihood ratio is an indicator to measure whether a signal exists or not. The probability hypothesis can be expressed as the following formula: (12) Using the original input signal, according to formulas (10), (11), (12), the likelihood ratio can be expressed as: (13) (14) The likelihood ratio function is constructed based on the signal's covariance matrix, noise correlation matrix, and noise power spectrum. In signal processing, by constructing such a likelihood ratio function, we can comprehensively consider various characteristics of the signal and noise as well as prior probability information, and more accurately judge the existence of the sound source from a probability perspective. When the likelihood ratio is greater than 1, it tends to be the existence of the sound source; when it is less than 1, it tends to be the non-existence of the sound source. It can use mathematical models to quantitatively judge the existence of the sound source in a complex acoustic environment, overcoming the limitations of simple threshold judgments.

[0064] Defined as the prior signal-to-noise ratio of the sound source q. Using the matrix inversion lemma, the input signal covariance matrix can be expressed as: (15) The related terms in the likelihood ratio function are further adjusted to be consistent with the previous parameters and operation results, so as to more accurately reflect the relationship between signal and noise at the covariance matrix level, thereby improving the accuracy of the likelihood ratio function in judging the existence of sound sources.

[0065] Using the above formula, substituting (14) and (15) into the likelihood ratio (13) can be expressed as: (16) Among them, , by calculating the power of the transient MVDR beam: (17) It reflects the signal energy distribution of the sound source while considering the noise correlation and sound source propagation characteristics. In a multi-sound source and complex noise environment, this power value can be used as an important indicator to distinguish the strength and position characteristics of different sound sources. When locating a sound source, if the power value corresponding to a sound source is large, it means that the signal strength of the sound source in the current environment is relatively strong, and a higher weight will be given in the positioning process, which helps to more accurately determine the location of the sound source. At the same time, it is also a key component when calculating the posterior signal-to-noise ratio. The posterior signal-to-noise ratio is further used to calculate the likelihood ratio function, so as to more accurately judge the probability of the existence of the sound source, provide an important basis for the entire signal processing process, and realize the effective identification and processing of sound sources in complex acoustic environments.

[0066] represents the channel matrix, which contains the channel gain and phase information from the sound source q to the microphone array. The method for obtaining the channel matrix is ​​well-known technology, and the specific acquisition process is not repeated here. is the conjugate transposed matrix of the channel matrix.

[0067] It can be the a posteriori signal-to-noise ratio of the sound source signal q: (18) Prior signal-to-noise ratio (based on the pre-estimation of the signal power spectrum of the sound source or based on previous experience, statistical analysis under specific circumstances) and the posterior signal-to-noise ratio (18) are substituted into formula (16), and the likelihood ratio function can be expressed as follows (19): (19) Compared with the traditional speech separation SPP algorithm that uses a fixed probability density function, the likelihood ratio function can obtain more accurate sound source localization statistics. The calculation results can be updated in real time according to the received signal to adapt to the dynamic changes of the acoustic environment. When a new sound source appears in the environment, the noise intensity or frequency characteristics change, or the user's head moves, the statistical characteristics of the signal will change.

[0068] The SPP algorithm for speech separation can calculate the dynamic existence probability in real time according to the current acoustic environment and the received signal, and can respond quickly to these changes. It uses the statistical characteristics of the signal and noise, considers the two hypotheses of the existence and non-existence of the sound source, and calculates the probability of each hypothesis for a given noise signal in these two cases, thereby providing a more dynamic and data-driven probability estimate for the existence of each sound source, making up for the shortcomings of the Gaussian mixture model GMM.

[0069] When calculating the probability of existence of each time-frequency frame, the speech separation SPP algorithm not only considers the signal characteristics of the current time-frequency frame, but also combines the signal changes of adjacent time-frequency frames. By analyzing the time series of the signal, it can more accurately capture the dynamic changes of the sound source and noise, thereby optimizing the probability calculation model. In addition, SPP also fully considers the uncertainty and complexity of noise, analyzes the statistical characteristics of noise, and introduces a more accurate noise model, so as to more effectively suppress the interference of noise in the probability calculation process and improve the accuracy and reliability of the estimation of the probability of the sound source.

[0070] By providing a dynamic probability of the existence of sound sources, the speech separation SPP algorithm can optimize the performance of the entire adaptive binaural beamformer system. It can more accurately adapt to the actual sound environment, thereby better achieving sound source separation and noise suppression, improving the overall performance of hearing aids in complex acoustic environments, and meeting users' needs for hearing assistance in different scenarios.

[0071] Among them, the speech separation SPP algorithm is a well-known technology, and its specific principle will not be repeated here.

[0072] Preferably, the sound source localization flow chart provided in this embodiment is as follows: Figure 2 shown.

[0073] At this point, the existence probability of the dynamic sound source and the posterior signal-to-noise ratio are obtained, which increases the accuracy of sound source localization.

[0074] Step S5: Based on the received signals of each microphone in the current time period, obtain the beam of the current time period; based on the dynamic existence probability of each sound source at the sound source angle of the current time period, determine the iteration step of each sound source at the sound source angle of the current time period, and combine the beam of the current time period and the received signals of each microphone to determine the adaptive noise elimination coefficient of each sound source in the next time period to enhance the sound source.

[0075] The generalized sidelobe canceller GSC in this example uses a fixed beamformer and adopts the minimum variance distortionless response (MVDR) method to generate a fixed beam. The expression of the beam output is: (20) in, represents the conjugate transpose of the head-related function of the qth sound source, represents the inverse of the noise correlation matrix, Represents the input microphone signal, and the denominator plays a role in normalization and weighting. It adjusts the input signal according to the direction of the sound source and the noise characteristics, so that when the beam is output, the signal in the direction of the target sound source can be enhanced, while the signal (including noise) in the non-target direction is suppressed. Compared with the traditional delay-and-add beam, this method of considering the time difference between the two ears is more sophisticated. In binaural hearing, due to the presence of the head, the time it takes for the sound to arrive at the two ears will be different. This feature can be used to more accurately locate the sound source and improve the noise suppression effect.

[0076] The adaptive blocking matrix coefficients of each sound source in the next time period are the microphone signals in the next time period and the dynamic existence probability of each sound source as the input of the generalized sidelobe canceller, and the output result is obtained. The specific acquisition principle of the adaptive blocking matrix coefficients is as follows: The generalized sidelobe canceller GSC in this embodiment uses an adaptive blocking matrix to suppress the sound in the target direction and generate noise and interference reference signals for noise cancellation. For the target sound source q, the target existence probability result estimated by the Gaussian mixture model is first used to construct the target signal projection matrix.

[0077] (twenty one) in, In the direction of the sound source Under the condition, the probability of the existence of sound source q is, It is the probability of the existence of a sound source calculated based on the signal-to-noise ratio. In a complex acoustic environment, the existence of a sound source, as well as its intensity and position, changes dynamically. Through the calculation of the speech separation SPP algorithm and the Gaussian mixture model GMM, the existence probability of each sound source in different time-frequency frames can be estimated more accurately. This probability value plays a key weighting role in the construction of the target signal projection matrix. When the probability is high, it means that the sound source in the current frame is more likely to exist. At this time, when constructing the projection matrix, it will be more inclined to use the information of the current frame to highlight the target sound source; conversely, when the probability is low, it will refer more to the projection matrix of the previous frame. , in order to maintain a certain stability and continuity.

[0078] Represents the microphone signal covariance matrix, which reflects the correlation and energy distribution of the signals received by the microphone between different channels. represents the energy of the input signal, As part of the calculation, the covariance matrix can be normalized so that signals of different energy levels can be properly processed in the projection matrix construction, avoiding processing deviations caused by large differences in signal energy.

[0079] Compute the complement space matrix of the target signal, which masks the target signal.

[0080] (twenty two) For the sound source q, the matrix parameters of the M-1th row and the Mth column of the complementary space matrix are selected to generate the adaptive blocking matrix coefficients. Indicates a The unit matrix has 1 elements on the diagonal and 0 elements on the rest. By subtracting the target signal projection matrix from the unit matrix, the target signal complementary space matrix obtained can shield the target signal. In subsequent processing, such as calculating the blocking matrix coefficients, the complementary space matrix plays a key role. It can extract the part orthogonal to the target signal, so as to better generate noise and interference reference signals. For example, in an environment with multiple sound sources and noise, the target signal complementary space matrix can exclude the space where the target sound source signal is located, so that subsequent processing can focus more on the analysis and processing of noise and other interference signals, providing a more accurate basis for noise elimination.

[0081] The generalized sidelobe canceller GSC in this embodiment uses an adaptive noise canceller, using the above-mentioned adaptive blocking matrix output as a reference and the fixed beam output as an input signal to further eliminate the noise and interference signals in the target direction. The adaptive noise canceller uses a normalized minimum mean square filter, and the filter coefficient update process is as follows: The microphone signal in the current time period is used as the input of the minimum variance distortion-free response algorithm, and the beam in the current time period is output.

[0082] Among them, the minimum variance distortion-free response algorithm is a well-known technology, and the specific principle will not be described in detail.

[0083] Adaptive noise elimination coefficient of sound source q in the next period The expression is: (twenty three) In the formula, represents the adaptive noise elimination coefficient of the sound source q in the current period, where: The initial value of is 0; Indicates the beam in the current period; Represents the adaptive blocking matrix coefficient of the sound source q in the next period; Indicates the microphone signal in the current period; Indicates taking the absolute value.

[0084] Among them, the expression of the iteration step length of each sound source under the sound source angle of the current period is: (twenty four) In the formula, Indicates the iteration step of the sound source q at the sound source angle of the current period; Indicates the probability of existence of sound source q under the sound source angle of the current period; Indicates a preset value.

[0085] It should be noted that The value range of is generally between 0 and 1. The value of is 0.5, and the implementer can also set it according to the specific situation. This embodiment does not impose any special limitation.

[0086] Furthermore, the microphone signal of the next time period is used as the input of the minimum distortion-free variance, and a preliminary enhanced signal is output, and the enhanced signal is used as the input of the generalized sidelobe cancellation technology, wherein the adaptive noise elimination coefficients of each sound source in the next time period are used as the adaptive filter coefficients in the adaptive filter of the generalized sidelobe canceller, and the final enhanced signal is output.

[0087] The working principle of the generalized sidelobe canceller is a well-known technology, and the specific principle will not be described in detail.

[0088] It should be noted that the above sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0089] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0090] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Modifications to the technical solutions recorded in the aforementioned embodiments, or equivalent replacement of some of the technical features therein, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A sound source localization and enhancement method for binaural hearing aids, characterized in that: The method comprises the following steps: There are multiple sound sources in the environment, and the receiving signals of each microphone within a preset time length are obtained, and the preset time length is divided into multiple time periods; According to the correlation between the received signals of different microphones at any time period, combined with random maximum likelihood estimation, the sound source angle at any time period is determined; At the sound source angle in any period of time, by analyzing the distribution of each sound source and combining it with the Gaussian mixture model, the existence probability of each sound source at the sound source angle in any period of time is determined; Based on the existence probability of each sound source at the sound source angle in any period of time, determine the dynamic existence probability of each sound source at the sound source angle in any period of time; Based on the received signals of each microphone in the current time period, the beam of the current time period is obtained; based on the dynamic existence probability of each sound source at the sound source angle of the current time period, the iteration step of each sound source at the sound source angle of the current time period is determined, and the beam of the current time period and the received signals of each microphone are combined to determine the adaptive noise elimination coefficient of each sound source in the next time period to enhance the sound source.

2. A method for sound source localization and enhancement for binaural hearing aids as claimed in claim 1, characterized in that: The method for determining the sound source angle in any period of time is: The set of received signals of all microphones in any period of time is taken as the microphone signal, and the covariance matrix of the microphone signal is obtained as the input of the random maximum likelihood estimation algorithm. The random maximum likelihood estimation algorithm is iterated to output the sound source angle in any period of time.

3. The method for sound source localization and enhancement for binaural hearing aids according to claim 1, characterized in that: The method for determining the existence probability of each sound source at the sound source angle in any period of time is: The expression of the existence probability of sound source q under the sound source angle of time period b is for: ; In the formula, represents the prior probability of the qth Gaussian component in the Gaussian mixture model, where one Gaussian component represents one sound source; represents the qth Gaussian component; represents the angle of the sound source in time period b; , Respectively represent the means of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model; , They represent the variance of the qth Gaussian component and the i-th Gaussian component in the Gaussian mixture model respectively; represents the Gaussian distribution function; Q represents the number of all Gaussian components.

4. A method for sound source localization and enhancement for binaural hearing aids as claimed in claim 3, characterized in that: The prior probability, mean and variance of the Gaussian component are estimated by the expectation maximization algorithm.

5. The method for sound source localization and enhancement for binaural hearing aids according to claim 1, characterized in that: The method for determining the dynamic existence probability of each sound source is: The existence probability of each sound source under the sound source angle in any period of time is used as the input of the speech separation SPP algorithm, and the dynamic existence probability of each sound source under the sound source angle in any period of time is output.

6. The method for sound source localization and enhancement for binaural hearing aids according to claim 3, characterized in that: The expression of the iteration step length of each sound source under the sound source angle of the current period is: ; In the formula, Indicates the iteration step of the sound source q at the sound source angle of the current period; Indicates the probability of existence of sound source q under the sound source angle of the current period; Indicates a preset value.

7. The method for sound source localization and enhancement for binaural hearing aids according to claim 2, characterized in that: The process of acquiring the beam in the current period is as follows: The microphone signal in the current time period is used as the input of the minimum variance distortion-free response algorithm, and the beam in the current time period is output.

8. The method for sound source localization and enhancement for binaural hearing aids according to claim 6, characterized in that: The step of determining the adaptive noise elimination coefficient of each sound source in the next time period includes: Adaptive noise elimination coefficient of sound source q in the next period The expression is: ; In the formula, represents the adaptive noise elimination coefficient of the sound source q in the current period, where: The initial value of is 0; Indicates the beam in the current period; Represents the adaptive blocking matrix coefficient of the sound source q in the next period; Indicates the microphone signal in the current period; Indicates taking the absolute value.

9. A method for sound source localization and enhancement for binaural hearing aids as claimed in claim 8, characterized in that: The adaptive blocking matrix coefficients of the sound source q in the next time period are the microphone signal in the next time period and the dynamic existence probability of the sound source q as the input of the generalized sidelobe canceller, and the output result is obtained.

10. The method for sound source localization and enhancement for binaural hearing aids according to claim 1, characterized in that: The step of enhancing the sound source comprises: The microphone signal of the next time period is used as the input of the minimum distortion-free variance, and a preliminary enhanced signal is output, and the enhanced signal is used as the input of the generalized sidelobe cancellation technology, wherein the adaptive noise elimination coefficients of each sound source in the next time period are used as the adaptive filter coefficients in the adaptive filter of the generalized sidelobe canceller, and the final enhanced signal is output.

Images