Hearing aid signal processing method and system based on electroencephalogram detection, medium and product

Abstract

The application discloses a kind of based on electroencephalogram detection's hearing aid signal processing method, system, medium and product, belong to hearing aid signal processing field, the method is: synchronous acquisition user's original electroencephalogram signal, microphone array signal, inertial measurement data and historical previous frame audio drive signal;The independent candidate sound source stream is obtained by blind source separation to microphone array signal, according to inertial measurement data and historical previous frame audio drive signal, original electroencephalogram signal is handled to motion artifact cancellation and audio interference cancellation, and electroencephalogram signal is obtained;The target sound source probability value is obtained by neural decoding matching analysis to independent candidate sound source stream and electroencephalogram signal, according to target sound source probability value and microphone array signal, spatial enhancement beam forming processing is carried out, and enhanced audio signal is obtained and output. Implement the present application, can solve the problem that existing hearing aid is introduced when electroencephalogram control is caused by physical interference and algorithm limitation electroencephalogram signal distortion and hearing spatial sense loss.

Description

Technical Field

[0001] This invention belongs to the field of hearing aid signal processing, and relates to a hearing aid signal processing method, system, medium and product based on electroencephalogram (EEG) detection. Background Technology

[0002] With the aging population and the growing number of people experiencing hearing loss, hearing aids, as a primary intervention method, have widely adopted digital signal processing technology to enhance speech. Simultaneously, the academic community has proposed auditory attention decoding technology, which aims to determine the speaker a user is paying attention to in a complex acoustic environment by decoding the user's electroencephalogram (EEG) signals, thereby achieving intelligent sound selection.

[0003] Existing high-end hearing aids typically utilize microphone arrays to capture sound, combining beamforming and noise reduction algorithms to enhance speech. Some attempt to incorporate auditory attention decoding technology for automatic program switching. However, when these technologies are actually implemented in in-ear hearing aids, the high-intensity alternating magnetic field and mechanical vibrations generated by the receiver directly couple to the EEG electrodes placed close to the ear canal, resulting in severe microphonic potentials and motion artifacts. This causes the acquired EEG signals to be completely obscured by noise and rendered ineffective. Furthermore, traditional neuro-guided beamforming technology often employs hard-switching logic, completely suppressing non-target sound sources. This approach disrupts the spatial cues of binaural hearing, causing users to hear the content clearly but lose their sense of sound source location, increasing the cognitive load on the brain and making it difficult to maintain stable closed-loop control. Summary of the Invention

[0004] This application provides a method, system, medium, and product for hearing aid signal processing based on electroencephalography (EEG) detection, which can solve the problems of EEG signal distortion and loss of auditory spatial sense caused by physical interference and algorithm limitations when introducing EEG control in existing hearing aids.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a hearing aid signal processing method based on electroencephalogram (EEG) detection, comprising:

[0006] The hearing aid simultaneously collects the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal from the previous historical frame.

[0007] Blind source separation is performed on the microphone array signal to obtain independent candidate sound source streams. Based on the inertial measurement data and the audio driving signal of the previous historical frame, motion artifact cancellation processing and audio interference cancellation processing are performed on the original EEG signal to obtain the EEG signal.

[0008] Based on the independent candidate sound source streams and EEG signals, neural decoding matching analysis is performed to obtain the target sound source probability value. Based on the target sound source probability value and the microphone array signal, spatial enhancement beamforming processing is performed to obtain an enhanced audio signal and output it.

[0009] Compared to existing technologies, the embodiments of this application have the following beneficial effects: By simultaneously acquiring raw EEG signals, microphone array signals, inertial measurement data, and the audio driving signal from the previous historical frame through a hearing aid, a foundation for multimodal data perception is constructed; blind source separation of the microphone array signals yields independent candidate sound source streams, enabling the deconstruction of the mixed sound field without prior sound source location information, providing independent candidate objects for attention decoding; simultaneously, based on the inertial measurement data and the audio driving signal from the previous historical frame, motion artifact cancellation and audio interference cancellation are sequentially performed on the raw EEG signals. The use of the audio driving signal from the previous historical frame as a reference is based on the physical delay in sound wave propagation from the receiver to the ear canal electrodes and the computational overhead of digital signal processing. The causal nature of time allows the system to accurately predict and cancel electromagnetic and micro-sound interference generated at the current moment using known "past" output signals, thus ensuring the feasibility of real-time noise reduction. Combined with the reference of inertial measurement data, it effectively cuts off the annihilation effect of receiver electromagnetic / mechanical interference and user head movement on weak EEG signals, improving the signal-to-noise ratio and usability of EEG signals. Then, based on independent candidate sound source streams and purified EEG signals, neural decoding matching analysis is performed to obtain the probability value of the target sound source, realizing active sound selection based on the user's true auditory intention. Finally, spatial enhancement beamforming processing is performed based on the target sound source probability value and microphone array signals to output enhanced audio signals, enhancing the target speech while preserving the spatial cues of background sound. The above steps work together to form a closed-loop control of "perception-decoding-enhancement", solving the technical problems of signal failure due to physical interference and loss of auditory spatial sense caused by traditional hard switching algorithms when introducing EEG control in existing hearing aids, and realizing an intelligent hearing aid experience that is both clear and has a natural spatial sense.

[0010] In some embodiments of the first aspect of this application, the step of performing blind source separation on the microphone array signal to obtain independent candidate sound source streams includes:

[0011] Based on the microphone array signal, the statistical domain feature matrix is ​​obtained by calculating the received signal correlation matrix and the noise correlation matrix.

[0012] Based on the statistical domain feature matrix, the acoustic steering vector of each independent sound source is identified by the generalized eigenvalue decomposition algorithm, and the spatial feature vector of the sound source is obtained.

[0013] Based on the spatial feature vector of the sound source, a mutually exclusive spatial filter weight vector that satisfies the linear constraints of unity gain and null trap is constructed to obtain the spatial filter parameters;

[0014] Based on the spatial filtering parameters, a linear filtering operation is performed on the microphone array signal to obtain the independent candidate sound source streams corresponding to potential sound sources in different directions.

[0015] Compared to existing technologies, the above embodiments have the following advantages: Calculating the correlation matrix of the received signal and the correlation matrix of the noise to obtain the statistical domain feature matrix quantifies the spatial statistical characteristics of multi-channel signals, providing a precise data foundation for sound source separation; further, using a generalized eigenvalue decomposition algorithm to identify the acoustic steering vector of each independent sound source to obtain the spatial feature vector of the sound source, and utilizing second-order statistical properties to accurately locate the directional features of independent sound sources without training; subsequently, constructing a mutually exclusive spatial filter weight vector that satisfies the unity gain and null linear constraints, mathematically ensuring that the filter completely suppresses interference from other directions while retaining the directional gain of the target sound source; performing linear filtering on the microphone array signal based on these spatial filtering parameters, ultimately obtaining the independent candidate sound source streams corresponding to potential sound sources in different directions, thus achieving high-quality blind source separation without prior information, providing a clean and independent acoustic reference object for the subsequent neural decoding module.

[0016] In some embodiments of the first aspect of this application, the step of performing motion artifact cancellation processing and audio interference cancellation processing on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal includes:

[0017] Based on the inertial measurement data, a first adaptive filter is constructed using a recursive least squares algorithm to estimate the motion artifact noise estimation signal related to the inertial measurement data and filter it out from the original EEG signal to obtain an intermediate EEG signal.

[0018] Based on the audio driving signal of the previous historical frame, a second adaptive filter is constructed using the normalized least mean square algorithm to estimate the audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and then filtered out in the intermediate state EEG signal to obtain the EEG signal.

[0019] Compared to existing technologies, the above embodiments have the following advantages: A first adaptive filter is constructed using a recursive least squares algorithm to estimate and filter out motion artifact noise estimation signals. This results in extremely fast convergence speed for non-stationary motion interference induced by inertial measurement data, effectively tracking the violent signal fluctuations generated by rapid head rotation and outputting intermediate-state EEG signals. Furthermore, a second adaptive filter is constructed using a normalized least mean square algorithm. Using the audio driving signal from the previous frame as a known reference input, it estimates and filters out audio interference noise estimation signals highly correlated with the current receiver operating state. The energy normalization mechanism overcomes the convergence instability problem caused by the large dynamic range of the receiver driving audio amplitude. This cascaded dual adaptive filter architecture specifically suppresses low-frequency baseline drift / contact impedance noise caused by mechanical motion and high-frequency electromagnetic / microphonic effects caused by receiver operation, significantly improving the purity and stability of the final output EEG signal and ensuring the reliability of EEG control during normal hearing aid amplification operation.

[0020] In some embodiments of the first aspect of this application, the step of constructing a first adaptive filter using a recursive least squares algorithm based on the inertial measurement data, estimating motion artifact noise estimation signals related to the inertial measurement data, and filtering them out from the original EEG signal to obtain an intermediate-state EEG signal includes:

[0021] Based on the duration of the original EEG signal, the discrete time step is initialized, and the inverse correlation matrix and filter weight vector of the recursive least squares algorithm are initialized.

[0022] Based on the inertial measurement data and the original EEG signal, a recursive least squares iterative process is performed until all discrete time steps are traversed, and the final intermediate EEG signal is output.

[0023] In each iteration, the triaxial acceleration and triaxial angular velocity data prior to the current time step are extracted from the inertial measurement data and a first reference vector is constructed. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the first reference vector. Combined with the current filter weight vector and the first reference vector, the motion artifact noise estimation signal for the current time step is estimated. The motion artifact noise estimation signal is subtracted from the original EEG signal for the current time step to obtain the intermediate-state EEG signal for the current time step, which is used as the prior error for this round. The current filter weight vector and inverse correlation matrix are updated based on the gain vector, the conjugate signal of the prior error, and the forgetting factor.

[0024] Compared with existing technologies, the above embodiments have the following advantages: by initializing the discrete time step and the inverse correlation matrix and filter weight vector of the recursive least squares algorithm, the initial state benchmark of the iterative processing is established; in each iteration, inertial measurement data is extracted to construct a reference vector containing triaxial acceleration and angular velocity, capturing the instantaneous and historical effects of motion; the gain vector is calculated using the current inverse correlation matrix, the preset forgetting factor, and the reference vector, where the forgetting factor is used to control the algorithm's forgetting speed of old data, enabling it to quickly adapt to non-stationary motion states such as rapid head rotation; the motion artifact noise estimation signal is estimated by combining the gain vector, the current weight vector, and the reference vector, and the estimated value is subtracted from the original signal to obtain the intermediate state EEG signal as the prior error, realizing real-time interference cancellation; finally, the weight vector and inverse correlation matrix are updated according to the gain vector, the conjugate signal of the prior error, and the forgetting factor, and the second-order statistical properties of the inverse correlation matrix are used to accelerate the convergence process, ensuring that the interference waveform can still be accurately fitted under violent motion and that effective low-frequency neural oscillation components are not mistakenly filtered out.

[0025] In some embodiments of the first aspect of this application, the step of constructing a second adaptive filter based on the audio driving signal of the previous historical frame using a normalized least mean square algorithm, estimating an audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and filtering it out in the intermediate state EEG signal to obtain the EEG signal includes:

[0026] Based on the duration of the intermediate-state EEG signal, the discrete time step is initialized, and the filter weight vector of the normalized least mean square algorithm is initialized.

[0027] Based on the audio driving signal of the previous historical frame and the intermediate EEG signal, normalized least mean square iteration processing is performed until all discrete time steps are traversed, and the final EEG signal is output.

[0028] In each iteration, audio data prior to the current time step is extracted from the previous frame's audio driving signal and a second reference vector is constructed. The second reference vector is then convolved with the current filter weight vector to estimate the audio interference noise signal at the current time step. The estimated audio interference noise signal is then filtered out from the intermediate EEG signal at the current time step to obtain the EEG signal at the current time step. The signal energy of the second reference vector is calculated, and the filter weight vector for the next round is updated using the signal energy, the EEG signal at the current time step, and the second reference vector, according to the normalized least mean square criterion.

[0029] Compared with existing technologies, the above embodiments have the following beneficial effects: By initializing the discrete time step and the filter weight vector of the normalized least mean square algorithm, the audio interference cancellation process is initiated; in each iteration, audio data before the current time step in the previous frame of the audio driving signal is extracted to construct a reference vector, locking in the known reference information of the interference source and using its time lag to match the current interference; the audio interference noise estimation signal is estimated by performing convolution operation on the reference vector using the current weight vector, simulating the physical path of the interference signal from the receiver to the EEG electrodes; the estimated value is filtered out from the intermediate state signal to directly obtain the pure EEG signal, achieving accurate cancellation in the time domain; the signal energy of the reference vector is calculated and the next round of weight vector is updated using this energy, the current EEG signal, and the reference vector. The introduction of signal energy realizes the normalization of the step size, preventing division by zero errors or divergence when the input signal energy is close to zero, enabling the algorithm to adapt to the dynamic adjustment of hearing aid gain or the drastic fluctuation of audio content, ensuring stable convergence of audio interference cancellation under various volume conditions.

[0030] In some embodiments of the first aspect of this application, the step of obtaining the target sound source probability value by performing neural decoding matching analysis based on the independent candidate sound source streams and EEG signals includes:

[0031] The independent candidate sound source streams are subjected to gamma-pass filtering and the amplitude envelope after power-law compression is extracted to obtain auditory envelope features that simulate the characteristics of the human cochlea.

[0032] The EEG signal was bandpass filtered to extract neural oscillation features covering the neural entrainment region and the attention control region;

[0033] The auditory envelope features and the neural oscillation features are input into a pre-trained convolutional neural network model, and the matching probability is calculated within a preset time window. The output is a target sound source probability value that indicates the degree to which the user pays attention to a specific speaker.

[0034] Compared to existing technologies, the above embodiments have the following beneficial effects: Gamma-pass filtering is applied to independent candidate sound source streams, and the amplitude envelope after power-law compression is extracted. This simulates the nonlinear frequency analysis and loudness compression characteristics of the human cochlea, obtaining auditory envelope features that conform to biological auditory mechanisms. Simultaneously, band-pass filtering is applied to EEG signals to extract neural oscillation features covering the neural entrainment region and the attention control region, filtering out low-frequency drift and high-frequency interference while retaining band information strongly correlated with auditory attention. The two types of features are input into a pre-trained convolutional neural network model, and the matching probability is calculated within a preset time window. The model's nonlinear mapping capability is used to capture the temporal correlation between EEG and the acoustic envelope, outputting a target sound source probability value indicating the degree to which the user focuses on a specific speaker. This achieves lightweight and high-precision real-time attention decoding, providing clear intentional instructions for subsequent sound field control.

[0035] In some embodiments of the first aspect of this application, the step of performing spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and outputting it includes:

[0036] The target sound source probability value is compared with a preset high and low threshold and the duration is determined to obtain the target sound source identifier. Based on the target sound source identifier, a smoothing gain coefficient is calculated in combination with a preset smoothing factor.

[0037] Based on the smoothing gain coefficient, the target sound source identifier, and the microphone array signal, a multi-channel Wiener filter optimization objective function containing noise reduction and spatial constraint terms is constructed, and the optimal weight vector is obtained by solving the problem.

[0038] The optimal weight vector is used to perform a linear filtering operation on the microphone array signal to obtain a spatial enhancement component, and the reference channel signal in the microphone array is obtained as the background component.

[0039] Based on the smoothing gain coefficient, spatial enhancement component, and background component, a weighted synthesis operation is performed to obtain the final enhanced audio signal and output it.

[0040] Compared with existing technologies, the above embodiments have the following beneficial effects: The target sound source probability value is compared with a preset high / low threshold and the duration is determined to obtain the target sound source identifier. Hysteresis logic is introduced to prevent misjudgment caused by short-term probability fluctuations. A smoothing gain coefficient is calculated using a smoothing factor, controlling the slope of gain changes and avoiding sudden auditory changes during sound source switching. A multi-channel Wiener filter containing noise reduction and spatial constraint terms is constructed to optimize the objective function and solve for the optimal weight vector, ensuring the preservation of spatial cues while enhancing speech. The optimal weight vector is used to filter the microphone array signal to obtain a spatial enhancement component, and a reference channel signal is obtained as the background component, separating the enhanced signal from the ambient noise. Finally, a weighted synthesis operation is performed based on the smoothing gain coefficient, the spatial enhancement component, and the background component. The slight preservation of non-target background sound prevents the loss of spatial sense caused by complete silence, ultimately outputting a clear, loud, and naturally directional enhanced audio signal, effectively reducing the user's cognitive load and maintaining a long-term auditory attention loop.

[0041] Secondly, the present invention also provides a hearing aid signal processing system based on electroencephalogram (EEG) detection, comprising: a data acquisition module, a preprocessing module, and a result output module;

[0042] The data acquisition module is used to synchronously collect the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal of the previous historical frame through the hearing aid.

[0043] The pre-processing module is used to perform blind source separation on the microphone array signal to obtain independent candidate sound source streams, and to perform motion artifact cancellation processing and audio interference cancellation processing on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal.

[0044] The result output module is used to perform neural decoding matching analysis based on the independent candidate sound source stream and EEG signal to obtain the target sound source probability value, and to perform spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and output it.

[0045] Compared to existing technologies, the above embodiments of this application have the following beneficial effects: By simultaneously acquiring raw EEG signals, microphone array signals, inertial measurement data, and the audio driving signal from the previous historical frame through a hearing aid, a foundation for multimodal data perception is constructed; blind source separation of the microphone array signals yields independent candidate sound source streams, enabling the deconstruction of the mixed sound field without prior sound source location information, providing independent candidate objects for attention decoding; simultaneously, based on the inertial measurement data and the audio driving signal from the previous historical frame, motion artifact cancellation and audio interference cancellation are sequentially performed on the raw EEG signals. The use of the audio driving signal from the previous historical frame as a reference is based on the physical delay in sound wave propagation from the receiver to the ear canal electrodes and the computational limitations of digital signal processing. The time-consuming causal nature of the system allows it to accurately predict and cancel electromagnetic and micro-sound interference generated at the current moment using known "past" output signals, thus ensuring the feasibility of real-time noise reduction. Combined with inertial measurement data, it effectively cuts off the annihilation effect of receiver electromagnetic / mechanical interference and user head movements on weak EEG signals, improving the signal-to-noise ratio and usability of EEG signals. Furthermore, it performs neural decoding matching analysis based on independent candidate sound source streams and purified EEG signals to obtain the target sound source probability value, enabling active sound selection based on the user's true auditory intent. Finally, it performs spatial enhancement beamforming processing based on the target sound source probability value and microphone array signals, outputting an enhanced audio signal that enhances the target speech while preserving spatial cues of background sound. These steps work synergistically to form a closed-loop control of "perception-decoding-enhancement," solving the technical challenges of signal failure due to physical interference and loss of auditory spatial sense caused by traditional hard-switching algorithms when introducing EEG control in existing hearing aids. This achieves a clear and naturally spatial intelligent hearing aid experience.

[0046] Thirdly, the present invention also provides a computer program product, including a computer program or instructions, which, when executed, implement any one of the hearing aid signal processing methods based on electroencephalogram (EEG) detection of the present invention.

[0047] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any one of the hearing aid signal processing methods based on electroencephalogram (EEG) detection of this invention. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating a hearing aid signal processing method based on electroencephalography (EEG) detection, provided in some embodiments of the present invention.

[0049] Figure 2 This is a schematic diagram of the structure of a hearing aid signal processing system based on electroencephalography (EEG) detection, provided in some embodiments of the present invention.

[0050] Figure 3 This is a flowchart illustrating a process architecture provided in some embodiments of the present invention.

[0051] Figure 4 This is a cross-sectional schematic diagram of a three-layer coaxial suspended electrode module provided in some embodiments of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1:

[0054] Please refer to Figure 1 To address the issues of EEG signal distortion and loss of auditory spatial perception caused by physical interference and algorithmic limitations when introducing EEG control in existing hearing aids, an embodiment of the present invention provides a hearing aid signal processing method based on EEG detection, comprising steps S1 to S3:

[0055] Step S1: Synchronously acquire the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal of the previous historical frame through the hearing aid.

[0056] Step S2: Perform blind source separation on the microphone array signal to obtain independent candidate sound source streams, and perform motion artifact cancellation processing and audio interference cancellation processing on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal.

[0057] Furthermore, the blind source separation can be implemented through the following preferred embodiments, including steps S21-S24, as follows:

[0058] S21: Based on the microphone array signal, calculate the received signal correlation matrix and noise correlation matrix to obtain the statistical domain feature matrix;

[0059] S22: Based on the statistical domain feature matrix, the acoustic steering vector of each independent sound source is identified by the generalized eigenvalue decomposition algorithm to obtain the sound source spatial feature vector;

[0060] S23: Based on the spatial feature vector of the sound source, construct a mutually exclusive spatial filter weight vector that satisfies the linear constraints of unity gain and null, and obtain the spatial filter parameters;

[0061] S24: Based on the spatial filtering parameters, perform a linear filtering operation on the microphone array signal to obtain the independent candidate sound source streams corresponding to potential sound sources in different directions.

[0062] In this preferred embodiment, the statistical domain feature matrix is ​​obtained by calculating the correlation matrix of the received signal and the noise correlation matrix, quantifying the spatial statistical characteristics of the multi-channel signal and providing an accurate data foundation for sound source separation. Furthermore, the acoustic steering vector of each independent sound source is identified using a generalized eigenvalue decomposition algorithm to obtain the spatial feature vector of the sound source. Second-order statistical properties are utilized to accurately locate the directional features of independent sound sources without training. Subsequently, a mutually exclusive spatial filter weight vector satisfying unity gain and null linear constraints is constructed, mathematically ensuring that the filter completely suppresses interference from other directions while retaining the directional gain of the target sound source. Based on these spatial filtering parameters, a linear filtering operation is performed on the microphone array signal, ultimately obtaining the independent candidate sound source streams corresponding to potential sound sources in different directions. This achieves high-quality blind source separation without prior information, providing a clean and independent acoustic reference object for the subsequent neural decoding module.

[0063] Furthermore, the motion artifact cancellation processing and audio interference cancellation processing can be implemented through the following preferred embodiments, including steps S25-S26:

[0064] S25: Based on the inertial measurement data, a first adaptive filter is constructed using a recursive least squares algorithm to estimate the motion artifact noise estimation signal related to the inertial measurement data and filter it out from the original EEG signal to obtain an intermediate EEG signal;

[0065] S26: Based on the audio driving signal of the previous historical frame, a second adaptive filter is constructed using the normalized least mean square algorithm to estimate the audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and then filtered out in the intermediate state EEG signal to obtain the EEG signal.

[0066] In this preferred embodiment, a first adaptive filter is constructed using a recursive least squares algorithm to estimate and filter out motion artifact noise estimation signals. This filter exhibits extremely fast convergence speed against non-stationary motion interference induced by inertial measurement data, effectively tracking the drastic signal fluctuations generated by rapid head rotation and outputting intermediate-state EEG signals. A second adaptive filter is further constructed using a normalized least mean square algorithm. Using the audio drive signal from the previous frame as a known reference input, this filter estimates and filters out audio interference noise estimation signals highly correlated with the current receiver operating state. An energy normalization mechanism overcomes the convergence instability problem caused by the large dynamic range of the receiver's drive audio amplitude. This cascaded dual adaptive filter architecture specifically suppresses low-frequency baseline drift / contact impedance noise caused by mechanical motion and high-frequency electromagnetic / microphonic effects caused by receiver operation, significantly improving the purity and stability of the final output EEG signal and ensuring the reliability of EEG control during normal hearing aid amplification operation.

[0067] Furthermore, step S25 can be implemented through the following preferred embodiments, including steps S251-S252, as follows:

[0068] S251: Based on the duration of the original EEG signal, initialize the discrete time step, and initialize the inverse correlation matrix and filter weight vector of the recursive least squares algorithm.

[0069] S252: Based on the inertial measurement data and the original EEG signal, perform recursive least squares iterative processing until all discrete time steps have been traversed, and output the final intermediate state EEG signal;

[0070] In each iteration, the triaxial acceleration and triaxial angular velocity data prior to the current time step are extracted from the inertial measurement data and a first reference vector is constructed. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the first reference vector. Combined with the current filter weight vector and the first reference vector, the motion artifact noise estimation signal for the current time step is estimated. The motion artifact noise estimation signal is subtracted from the original EEG signal for the current time step to obtain the intermediate-state EEG signal for the current time step, which is used as the prior error for this round. The current filter weight vector and inverse correlation matrix are updated based on the gain vector, the conjugate signal of the prior error, and the forgetting factor.

[0071] In this preferred embodiment, the initial state benchmark for iterative processing is established by initializing the discrete time step and the inverse correlation matrix and filter weight vector of the recursive least squares algorithm. In each iteration, inertial measurement data is extracted to construct a reference vector containing triaxial acceleration and angular velocity, capturing the instantaneous and historical effects of motion. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the reference vector, where the forgetting factor controls the algorithm's forgetting speed of old data, enabling it to quickly adapt to non-stationary motion states such as rapid head rotation. The motion artifact noise estimation signal is estimated by combining the gain vector, the current weight vector, and the reference vector, and the estimated value is subtracted from the original signal to obtain the intermediate EEG signal as the prior error, achieving real-time interference cancellation. Finally, the weight vector and inverse correlation matrix are updated according to the gain vector, the conjugate signal of the prior error, and the forgetting factor. The second-order statistical properties of the inverse correlation matrix are used to accelerate the convergence process, ensuring that the interference waveform can still be accurately fitted under violent motion without filtering out effective low-frequency neural oscillation components.

[0072] Furthermore, step S26 can be implemented through the following preferred embodiments, including steps S261-S262, as follows:

[0073] S261: Based on the duration of the intermediate-state EEG signal, initialize the discrete time step and initialize the filter weight vector of the normalized least mean square algorithm.

[0074] S262: Based on the audio driving signal of the previous historical frame and the intermediate EEG signal, perform normalized least mean square iteration processing until all discrete time steps have been traversed, and output the final EEG signal.

[0075] In each iteration, audio data prior to the current time step is extracted from the previous frame's audio driving signal and a second reference vector is constructed. The second reference vector is then convolved with the current filter weight vector to estimate the audio interference noise signal at the current time step. The estimated audio interference noise signal is then filtered out from the intermediate EEG signal at the current time step to obtain the EEG signal at the current time step. The signal energy of the second reference vector is calculated, and the filter weight vector for the next round is updated using the signal energy, the EEG signal at the current time step, and the second reference vector, according to the normalized least mean square criterion.

[0076] In this preferred embodiment, the audio interference cancellation process is initiated by initializing the discrete time step and the filter weight vector of the normalized least mean square algorithm. In each iteration, audio data before the current time step in the previous frame of the audio driving signal is extracted to construct a reference vector, locking in the known reference information of the interference source and using its time lag to match the current interference. The audio interference noise estimation signal is estimated by convolving the reference vector with the current weight vector, simulating the physical path of the interference signal from the receiver to the EEG electrodes. The estimated value is filtered out from the intermediate state signal to directly obtain the pure EEG signal, achieving accurate cancellation in the time domain. The signal energy of the reference vector is calculated and the weight vector of the next round is updated using this energy, the current EEG signal, and the reference vector. The introduction of signal energy realizes the normalization of the step size, preventing division by zero errors or divergence when the input signal energy is close to zero. This allows the algorithm to adapt to the dynamic adjustment of hearing aid gain or the drastic fluctuation of audio content, ensuring stable convergence of audio interference cancellation under various volume conditions.

[0077] Step S3: Based on the independent candidate sound source stream and EEG signal, perform neural decoding matching analysis to obtain the target sound source probability value, and based on the target sound source probability value and the microphone array signal, perform spatial enhancement beamforming processing to obtain an enhanced audio signal and output it.

[0078] Furthermore, in step S3, the neural decoding matching analysis can be implemented through the following preferred embodiments, including steps S31-S33:

[0079] S31: Perform gamma-pass filtering on the independent candidate sound source streams and extract the amplitude envelope after power-law compression to obtain auditory envelope features that simulate the characteristics of the human cochlea.

[0080] S32: Bandpass filter the EEG signal to extract neural oscillation features covering the neural entrainment region and the attention control region;

[0081] S33: Input the auditory envelope features and the neural oscillation features into a pre-trained convolutional neural network model, calculate the matching probability within a preset time window, and output the target sound source probability value that indicates the degree to which the user pays attention to a specific speaker.

[0082] In this preferred embodiment, gamma-pass filtering is applied to independent candidate sound source streams, and the amplitude envelope after power-law compression is extracted. This simulates the nonlinear frequency analysis and loudness compression characteristics of the human cochlea, resulting in auditory envelope features that conform to biological auditory mechanisms. Simultaneously, band-pass filtering is applied to EEG signals to extract neural oscillation features covering the neural entrainment region and the attention control region, filtering out low-frequency drift and high-frequency interference while retaining band information strongly correlated with auditory attention. The two types of features are input into a pre-trained convolutional neural network model, and the matching probability is calculated within a preset time window. The model's nonlinear mapping capability is used to capture the temporal correlation between EEG and the acoustic envelope, outputting a target sound source probability value that indicates the degree to which the user pays attention to a specific speaker. This achieves lightweight and high-precision real-time attention decoding, providing clear intention instructions for subsequent sound field control.

[0083] Furthermore, in step S3, the spatial enhancement beamforming process can be implemented through the following preferred embodiments, including steps S34-S37:

[0084] S34: Compare the target sound source probability value with a preset high and low threshold and determine the duration to obtain the target sound source identifier, and calculate the smoothing gain coefficient based on the target sound source identifier and a preset smoothing factor.

[0085] S35: Based on the smoothing gain coefficient, the target sound source identifier, and the microphone array signal, construct a multi-channel Wiener filter optimization objective function that includes a noise reduction term and a spatial constraint term, and solve for the optimal weight vector;

[0086] S36: Perform a linear filtering operation on the microphone array signal using the optimal weight vector to obtain the spatial enhancement component, and obtain the reference channel signal in the microphone array as the background component;

[0087] S37: Based on the smoothing gain coefficient, spatial enhancement component, and background component, perform a weighted synthesis operation to obtain the final enhanced audio signal and output it.

[0088] In this preferred embodiment, the target sound source probability value is compared with a preset high and low threshold and the duration is determined to obtain the target sound source identifier. Hysteresis logic is introduced to prevent misjudgment caused by short-term fluctuations in probability. The smoothing gain coefficient is calculated by combining a smoothing factor to control the slope of gain change and avoid sudden changes in auditory perception when switching sound sources. A multi-channel Wiener filter containing noise reduction and spatial constraint terms is constructed to optimize the objective function and solve for the optimal weight vector, which ensures the preservation of spatial cues while enhancing speech. The microphone array signal is filtered using the optimal weight vector to obtain the spatial enhancement component, and the reference channel signal is obtained as the background component, separating the enhanced signal from the ambient noise. Finally, a weighted synthesis operation is performed based on the smoothing gain coefficient, the spatial enhancement component, and the background component. The slight preservation of non-target background sound prevents the loss of spatial sense caused by complete silence. The final output is an enhanced audio signal that is both clear and loud and has a natural sense of direction, effectively reducing the user's cognitive load and maintaining a long-term auditory attention loop.

[0089] In specific implementation, refer to Figure 3 The diagram illustrates a process architecture where Mic1 and Mic2 refer to the microphones of the left hearing aid, and Mic3 and Mic4 refer to the microphones of the right hearing aid. These four microphones together form a microphone array. After the hearing aids collect external sound data, EEG data, and data from the inertial measurement unit, to achieve active sound selection and enhancement based on the user's actual auditory intent while maintaining a natural auditory experience, this solution adopts a dual-path architecture of "candidate stream generation (bypass) + spatial enhancement (main path)," as detailed below:

[0090] To deconstruct the mixed sound field and provide independent "candidate objects" for AI without prior target orientation information (such as azimuth angle or voiceprint), blind source separation (BSS) and candidate stream generation are performed at the acoustic front end (bypass):

[0091] First, the multi-channel microphone signals are analyzed in the statistical domain, and the correlation matrix of the received signals from the microphone array is calculated. Correlation matrix with noise Then, by applying generalized eigenvalue decomposition, the number of independent sound sources is identified from the mixed signal subspace, for example, the main sound source is detected. and Extract the acoustic steering vector corresponding to each sound source, denoted as... (correspond )and (correspond These vectors describe the transfer function characteristics of different sound sources to the microphone array.

[0092] Next, mutually exclusive spatial filters are constructed. Based on the acoustic steering vector, two sets of linear spatial filter (beamformer) weight vectors are built. and To achieve sound source separation, the weight vector must satisfy the following linear constraints of unity gain and null:

[0093] ; Where 1 represents unity gain (retention) and 0 represents null (suppression).

[0094] Next, linear extraction of independent candidate streams is performed, and the constructed spatial filter is then used. and Each is applied to the original multi-channel mixed signal Above, perform a linear filtering operation:

[0095] ; ; thus generated and These are the independent candidate sound source streams. These two signals are then fed into the subsequent neural decoding module for attention comparison.

[0096] Furthermore, to address the unique interference sources in the hearing aid ear canal environment, this invention utilizes multi-sensor fusion technology to construct a dual adaptive filtering architecture for purifying electroencephalogram (EEG) signals:

[0097] 1. Motion artifact cancellation based on multidimensional IMU data and RLS algorithm:

[0098] Technical Principle: Hearing aids incorporate an inertial measurement unit (IMU) that monitors the user's head movements and body vibrations in real time. When head rotation or body vibration occurs, the electrodes and ear canal skin undergo minute displacements, leading to potential fluctuations at the electrode-skin contact interface, contact impedance changes, triboelectric noise, and low-frequency baseline drift. These interference signals are physically induced directly by mechanical motion and therefore highly correlated with the motion data acquired by the IMU, but uncorrelated with the electrical activity (EEG) generated by the brain. Based on this statistical characteristic, this invention employs an adaptive interference cancellation (AIC) architecture, utilizing a recursive least squares (RLS) algorithm to construct a "motion-artifact" transmission model to estimate and filter out non-stationary motion interference components in the EEG signal, as detailed below:

[0099] Based on the duration of the brainwave signal, let For discrete-time indexing, construct a multi-input single-output (MISO) adaptive filter, where the input data includes:

[0100] The main input signal, i.e., the real EEG signal acquired by the in-ear dry electrode, is... and motion artifact interference The original mixed signal.

[0101] The reference input vector is composed of multi-axis motion data output from the IMU sensor. To capture the instantaneous and historical effects of motion, a [defined...] is defined. For including triaxial acceleration and triaxial angular velocity The present and the past Data combinations at each time point:

[0102] ;in The total dimension of the reference signal.

[0103] Next, an adaptive filter is constructed using the Recursive Least Squares (RLS) algorithm, and iterative calculations are performed in the following order to estimate interference components and update the filter state in real time:

[0104] Calculate the gain vector :

[0105] ;in, It is the inverse correlation matrix. Forgetting factor, range of values (Recommended value) This is used to control the rate at which the algorithm forgets old data, in order to quickly track non-stationary motion states such as rapid head rotation.

[0106] Extracting EEG signals after removing motion artifacts (calculating prior error):

[0107] ;in, The intermediate state EEG signal after removing motion artifacts; It is a weight vector.

[0108] Update filter weight vector and the inverse correlation matrix:

[0109] : ;

[0110] ;in, It is the conjugate of the error signal.

[0111] By utilizing the RLS algorithm described above, this application achieves faster convergence speed compared to the traditional LMS algorithm by leveraging the second-order statistical properties of the input signal. When a user performs violent or sudden head movements, it can accurately fit the waveforms of low-frequency baseline drift and contact potential fluctuations, thereby effectively filtering out large-amplitude motion artifacts while protecting the Delta and Theta waves in the low-frequency range (0.5Hz - 4Hz) from being mistakenly filtered out.

[0112] 2. Audio interference cancellation based on AEC and NLMS algorithms.

[0113] Technical Principle: This invention addresses stimulus-related artifacts caused by high sound pressure level operation of the receiver, primarily including near-field electromagnetic induction artifacts and audio microphonic effect artifacts. Since these two types of artifacts are highly correlated with the receiver's driving audio signal in both the time and frequency domains, this invention utilizes the receiver's driving audio signal as a known reference and employs a Normalized Least Mean Squares (NLMS) adaptive filtering algorithm for linear cancellation. The algorithm implementation steps are as follows:

[0114] The input data is:

[0115] : The main input signal is the intermediate EEG signal after removing motion artifacts.

[0116] : Reference input vector, which is based on the digital audio drive signal sent to the receiver. The constructed delay vector has a length of .

[0117] Next, to address the near-field electromagnetic induction and microphonic effects caused by the receiver's high sound pressure level operation, a Normalized Least Mean Square (NLMS) algorithm is employed, including:

[0118] Interference signal estimation: using the current adaptive filter weight vector By performing a convolution operation on the reference input vector, the audio interference component superimposed on the EEG signal at the current moment is estimated. :

[0119] ;

[0120] Pure EEG output is represented as: ;in, This is the final output of a clean EEG signal.

[0121] Filter weight update (NLMS rule): Based on the NLMS criterion, the weights are iteratively updated using the normalized reference signal energy to adapt to changes in hearing aid gain or fluctuations in audio content.

[0122] ;in, This is the step size factor, used to control the convergence speed; This is a regularization parameter to prevent the input signal energy from being affected. A division-by-zero error occurs when the value is close to zero.

[0123] Using the NLMS algorithm, this application can track and lock the transfer function from the receiver audio signal to the EEG electrodes in real time. Compared to the standard LMS algorithm, NLMS overcomes the convergence instability problem caused by the large dynamic range of the audio signal amplitude through energy normalization. Even under dynamic adjustment of hearing aid gain or drastic fluctuations in audio content, the system can accurately estimate and subtract artifacts. This ensures that a high signal-to-noise ratio, pure EEG signals are obtained while the hearing aid is amplifying normally. .

[0124] After obtaining independent candidate sound source streams and clean EEG signals, lightweight attention decisions are made through a neural decoding terminal (bypass):

[0125] Feature alignment: for candidate streams and Gammatone filtering is performed and the amplitude envelope after power-law compression is extracted. The auditory envelope is then extracted by simulating the characteristics of the human cochlea. Meanwhile, the purified EEG signal Perform bandpass filtering from 1Hz to 32Hz to extract neural oscillation features. This frequency band covers Delta / Theta waves (neural entrainment region) and Alpha / Beta waves (attention control region), while filtering out low-frequency drift and high-frequency interference.

[0126] Similarity determination: Utilizing a pre-trained lightweight convolutional neural network (CNN) model. The matching probability between the EEG signal and the envelope of each candidate stream is calculated within a short time window (e.g., 1s-2s).

[0127] For example Output continuous probability values Instructing users to pay attention The degree of.

[0128] Next, in order to solve the problems of "abrupt auditory perception" and "loss of spatial sense" and achieve "natural auditory perception", it is necessary to preserve spatial cues and perform soft switching:

[0129] The input data includes:

[0130] Attention probability.

[0131] : The original multi-channel microphone signal.

[0132] : The target sound source guidance vector currently determined.

[0133] The calculation process includes:

[0134] First, the hysteresis and smoothing gain are calculated:

[0135] Hysteresis logic: Define state variables .like (e.g., 0.7) and duration (e.g., 500ms), the target is determined as ;like (e.g., 0.3) and duration The target was determined to be .

[0136] Smoothing gain: ;

[0137] in, For smoothing gain coefficient; A smoothing factor (e.g., 0.95) is used to control the slope of gain changes and prevent sudden changes in auditory perception. For indicator functions (if the target is) Output 1 if the output is 1, otherwise output 0.

[0138] Next, filter optimization based on the space-constrained cost function:

[0139] Construct the optimization objective function for the multi-channel Wiener filter (MWF), including noise reduction and spatial constraint terms:

[0140] ;

[0141] in, For noise reduction and enhancement; For spatial positioning constraints, The desired signal (i.e., the pure component of the target sound source at the reference microphone). The original head-related transfer function of the target sound source relative to the reference microphone; These are Lagrange multipliers used to adjust the weights for preserving spatial cues. The optimal weights can be obtained by analytically solving this function. .

[0142] Finally, a weighted composite output is performed:

[0143] Final output signal Spatial augmentation component and background component are synthesized by smooth gain weighting:

[0144] ;in, Reference microphone signal; The retention factor for non-target background noise (used to prevent the loss of spatial sense caused by complete silence); This is the final enhanced audio signal used to drive the receiver.

[0145] In addition, as a supplement, see reference Figure 4 The diagram shows a cross-sectional view of a three-layer coaxial suspended electrode module. To address the interference of the receiver's magnetic field on weak EEG signals in RIC hearing aids and the poor contact caused by ear canal deformation, and to achieve high signal-to-noise ratio and comfortable, unobtrusive EEG acquisition, this invention also designs a "three-layer coaxial suspended architecture," comprising:

[0146] 1) Inner core shielding layer: A high-permeability alloy shielding layer (such as permalloy) is wrapped around the receiver to cut off the coupling of electromagnetic radiation generated by the receiver during operation to weak brain signals from the source.

[0147] 2) Intermediate damping layer: A flexible damping dielectric layer (which can be an air gap, liquid silicone, or low-density foam material) is placed between the shielding layer and the outer electrode. This layer acts as a mechanical low-pass filter, effectively blocking the transmission of high-frequency mechanical vibrations from the receiver, achieving a "floating" effect on the electrodes. At the same time, this flexible layer also provides cushioning for the ear canal skin, improving wearing comfort.

[0148] 3) The outer skin-adaptive dry electrode layer is an elastomer that, when inserted into the ear canal, generates appropriate radial tension, ensuring that the dry electrode contacts fit snugly and comfortably against the ear canal skin. When the ear canal dynamically deforms due to the user's jaw movements (such as chewing or speaking), the elastomer's adaptive deformation capability, combined with the buffering effect of the intermediate damping layer, maintains a relatively constant contact pressure between the dry electrode and the skin, thereby significantly suppressing motion artifacts and ensuring the stability of signal acquisition.

[0149] By employing a three-layer coaxial suspension architecture consisting of an inner core shielding layer, a middle damping layer, and an outer skin adaptive dry electrode layer, this invention can simultaneously address the problem of strong near-field interference to weak EEG signals during the operation of RIC hearing aid receivers from both electromagnetic and mechanical perspectives within a very small space, thus ensuring the quality of EEG signal acquisition.

[0150] In summary, compared with the prior art, the above embodiments of this application have the following beneficial effects: By simultaneously acquiring raw EEG signals, microphone array signals, inertial measurement data, and the audio driving signal from the previous historical frame through a hearing aid, a foundation for multimodal data perception is constructed; blind source separation of the microphone array signals yields independent candidate sound source streams, enabling the deconstruction of the mixed sound field without prior sound source location information, providing independent candidate objects for attention decoding; simultaneously, based on the inertial measurement data and the audio driving signal from the previous historical frame, motion artifact cancellation processing and audio interference cancellation processing are sequentially performed on the raw EEG signals. The use of the audio driving signal from the previous historical frame as a reference is based on the physical delay in sound wave propagation from the receiver to the ear canal electrodes and the computational delay in digital signal processing. The causal nature of computational time allows the system to accurately predict and cancel electromagnetic and micro-sound interference generated at the current moment using known "past" output signals, thus ensuring the feasibility of real-time noise reduction. Combined with the reference of inertial measurement data, it effectively cuts off the annihilation effect of receiver electromagnetic / mechanical interference and user head movement on weak EEG signals, improving the signal-to-noise ratio and usability of EEG signals. Then, based on independent candidate sound source streams and purified EEG signals, neural decoding matching analysis is performed to obtain the probability value of the target sound source, realizing active sound selection based on the user's true auditory intention. Finally, spatial enhancement beamforming processing is performed based on the target sound source probability value and microphone array signals to output enhanced audio signals, enhancing the target speech while preserving the spatial cues of background sound. The above steps work together to form a closed-loop control of "perception-decoding-enhancement," solving the technical problems of signal failure due to physical interference and loss of auditory spatial sense caused by traditional hard switching algorithms when introducing EEG control in existing hearing aids, and achieving a clear and natural spatial sense of intelligent hearing aid experience.

[0151] Example 2:

[0152] Please refer to Figure 2 Based on the same inventive concept, the present invention discloses a hearing aid signal processing system based on electroencephalogram (EEG) detection, comprising: a data acquisition module M1, a preprocessing module M2, and a result output module M3;

[0153] The data acquisition module M1 is used to synchronously acquire the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal of the previous historical frame through the hearing aid.

[0154] The preprocessing module M2 is used to perform blind source separation on the microphone array signal to obtain independent candidate sound source streams, and to perform motion artifact cancellation processing and audio interference cancellation processing on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal.

[0155] Furthermore, the preprocessing module M2 includes: a matrix generation unit, a feature decomposition unit, a filter parameter generation unit, and a linear filtering unit;

[0156] The matrix generation unit is used to calculate the received signal correlation matrix and the noise correlation matrix to obtain the statistical domain feature matrix based on the microphone array signal.

[0157] The feature decomposition unit is used to identify the acoustic steering vector of each independent sound source based on the statistical domain feature matrix using a generalized eigenvalue decomposition algorithm, thereby obtaining the sound source spatial feature vector.

[0158] The filter parameter generation unit is used to construct a mutually exclusive spatial filter weight vector that satisfies the unity gain and null linearity constraints based on the sound source spatial feature vector, thereby obtaining spatial filter parameters.

[0159] The linear filtering unit is used to perform a linear filtering operation on the microphone array signal based on the spatial filtering parameters to obtain independent candidate sound source streams corresponding to potential sound sources in different directions.

[0160] In this preferred embodiment, the statistical domain feature matrix is ​​obtained by calculating the correlation matrix of the received signal and the noise correlation matrix, quantifying the spatial statistical characteristics of the multi-channel signal and providing an accurate data foundation for sound source separation. Furthermore, the acoustic steering vector of each independent sound source is identified using a generalized eigenvalue decomposition algorithm to obtain the spatial feature vector of the sound source. Second-order statistical properties are utilized to accurately locate the directional features of independent sound sources without training. Subsequently, a mutually exclusive spatial filter weight vector satisfying unity gain and null linear constraints is constructed, mathematically ensuring that the filter completely suppresses interference from other directions while retaining the directional gain of the target sound source. Based on these spatial filtering parameters, a linear filtering operation is performed on the microphone array signal, ultimately obtaining the independent candidate sound source streams corresponding to potential sound sources in different directions. This achieves high-quality blind source separation without prior information, providing a clean and independent acoustic reference object for the subsequent neural decoding module.

[0161] Furthermore, the preprocessing module M2 also includes: a first filtering unit and a second filtering unit;

[0162] The first filtering unit is used to construct a first adaptive filter based on the inertial measurement data using a recursive least squares algorithm, estimate the motion artifact noise estimation signal related to the inertial measurement data, and filter it out from the original EEG signal to obtain an intermediate EEG signal.

[0163] The second filtering unit is used to construct a second adaptive filter based on the audio driving signal of the previous historical frame using the normalized least mean square algorithm, estimate the audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and filter it out in the intermediate state EEG signal to obtain the EEG signal.

[0164] In this preferred embodiment, a first adaptive filter is constructed using a recursive least squares algorithm to estimate and filter out motion artifact noise estimation signals. This filter exhibits extremely fast convergence speed against non-stationary motion interference induced by inertial measurement data, effectively tracking the drastic signal fluctuations generated by rapid head rotation and outputting intermediate-state EEG signals. A second adaptive filter is further constructed using a normalized least mean square algorithm. Using the audio drive signal from the previous frame as a known reference input, this filter estimates and filters out audio interference noise estimation signals highly correlated with the current receiver operating state. An energy normalization mechanism overcomes the convergence instability problem caused by the large dynamic range of the receiver's drive audio amplitude. This cascaded dual adaptive filter architecture specifically suppresses low-frequency baseline drift / contact impedance noise caused by mechanical motion and high-frequency electromagnetic / microphonic effects caused by receiver operation, significantly improving the purity and stability of the final output EEG signal and ensuring the reliability of EEG control during normal hearing aid amplification operation.

[0165] Furthermore, the first filtering unit includes: a first initialization subunit and a first iteration subunit;

[0166] The first initialization subunit is used to initialize the discrete time step based on the duration of the original EEG signal, and to initialize the inverse correlation matrix and filter weight vector of the recursive least squares algorithm.

[0167] The first iteration unit is used to perform recursive least squares iterative processing based on the inertial measurement data and the original EEG signal until all discrete time steps have been traversed, and output the final intermediate EEG signal.

[0168] In each iteration, the triaxial acceleration and triaxial angular velocity data prior to the current time step are extracted from the inertial measurement data and a first reference vector is constructed. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the first reference vector. Combined with the current filter weight vector and the first reference vector, the motion artifact noise estimation signal for the current time step is estimated. The motion artifact noise estimation signal is subtracted from the original EEG signal for the current time step to obtain the intermediate-state EEG signal for the current time step, which is used as the prior error for this round. The current filter weight vector and inverse correlation matrix are updated based on the gain vector, the conjugate signal of the prior error, and the forgetting factor.

[0169] In this preferred embodiment, the initial state benchmark for iterative processing is established by initializing the discrete time step and the inverse correlation matrix and filter weight vector of the recursive least squares algorithm. In each iteration, inertial measurement data is extracted to construct a reference vector containing triaxial acceleration and angular velocity, capturing the instantaneous and historical effects of motion. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the reference vector, where the forgetting factor controls the algorithm's forgetting speed of old data, enabling it to quickly adapt to non-stationary motion states such as rapid head rotation. The motion artifact noise estimation signal is estimated by combining the gain vector, the current weight vector, and the reference vector, and the estimated value is subtracted from the original signal to obtain the intermediate EEG signal as the prior error, achieving real-time interference cancellation. Finally, the weight vector and inverse correlation matrix are updated according to the gain vector, the conjugate signal of the prior error, and the forgetting factor. The second-order statistical properties of the inverse correlation matrix are used to accelerate the convergence process, ensuring that the interference waveform can still be accurately fitted under violent motion without filtering out effective low-frequency neural oscillation components.

[0170] Furthermore, the second filtering unit includes: a second initialization subunit and a second iteration subunit;

[0171] The second initialization subunit is used to initialize the discrete time step and the filter weight vector of the normalized least mean square algorithm based on the duration of the intermediate state EEG signal.

[0172] The second iterative subunit is used to perform normalized least mean square iterative processing based on the audio driving signal of the previous historical frame and the intermediate EEG signal until all discrete time steps have been traversed, and output the final EEG signal.

[0173] In each iteration, audio data prior to the current time step is extracted from the previous frame's audio driving signal and a second reference vector is constructed. The second reference vector is then convolved with the current filter weight vector to estimate the audio interference noise signal at the current time step. The estimated audio interference noise signal is then filtered out from the intermediate EEG signal at the current time step to obtain the EEG signal at the current time step. The signal energy of the second reference vector is calculated, and the filter weight vector for the next round is updated using the signal energy, the EEG signal at the current time step, and the second reference vector, according to the normalized least mean square criterion.

[0174] In this preferred embodiment, the audio interference cancellation process is initiated by initializing the discrete time step and the filter weight vector of the normalized least mean square algorithm. In each iteration, audio data before the current time step in the previous frame of the audio driving signal is extracted to construct a reference vector, locking in the known reference information of the interference source and using its time lag to match the current interference. The audio interference noise estimation signal is estimated by convolving the reference vector with the current weight vector, simulating the physical path of the interference signal from the receiver to the EEG electrodes. The estimated value is filtered out from the intermediate state signal to directly obtain the pure EEG signal, achieving accurate cancellation in the time domain. The signal energy of the reference vector is calculated and the weight vector of the next round is updated using this energy, the current EEG signal, and the reference vector. The introduction of signal energy realizes the normalization of the step size, preventing division by zero errors or divergence when the input signal energy is close to zero. This allows the algorithm to adapt to the dynamic adjustment of hearing aid gain or the drastic fluctuation of audio content, ensuring stable convergence of audio interference cancellation under various volume conditions.

[0175] The result output module M3 is used to perform neural decoding matching analysis based on the independent candidate sound source stream and EEG signal to obtain the target sound source probability value, and to perform spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and output it.

[0176] Furthermore, the result output module M3 includes: a first feature extraction unit, a second feature extraction unit, and a model processing unit;

[0177] The first feature extraction unit is used to perform gamma-pass filtering on the independent candidate sound source stream and extract the amplitude envelope after power-law compression to obtain auditory envelope features that simulate the characteristics of the human cochlea.

[0178] The second feature extraction unit is used to perform bandpass filtering on the electroencephalogram (EEG) signal and extract neural oscillation features covering the neural entrainment region and the attention control region.

[0179] The model processing unit is used to input the auditory envelope features and the neural oscillation features into a pre-trained convolutional neural network model, calculate the matching probability within a preset time window, and output the target sound source probability value that indicates the degree to which the user pays attention to a specific speaker.

[0180] In this preferred embodiment, gamma-pass filtering is applied to independent candidate sound source streams, and the amplitude envelope after power-law compression is extracted. This simulates the nonlinear frequency analysis and loudness compression characteristics of the human cochlea, resulting in auditory envelope features that conform to biological auditory mechanisms. Simultaneously, band-pass filtering is applied to EEG signals to extract neural oscillation features covering the neural entrainment region and the attention control region, filtering out low-frequency drift and high-frequency interference while retaining band information strongly correlated with auditory attention. The two types of features are input into a pre-trained convolutional neural network model, and the matching probability is calculated within a preset time window. The model's nonlinear mapping capability is used to capture the temporal correlation between EEG and the acoustic envelope, outputting a target sound source probability value that indicates the degree to which the user pays attention to a specific speaker. This achieves lightweight and high-precision real-time attention decoding, providing clear intention instructions for subsequent sound field control.

[0181] Furthermore, the result output module M3 also includes: a coefficient calculation unit, an optimization unit, a component acquisition unit, and a fusion unit;

[0182] The coefficient calculation unit is used to compare the target sound source probability value with a preset high and low threshold and determine the duration to obtain the target sound source identifier, and calculate the smoothing gain coefficient based on the target sound source identifier and a preset smoothing factor.

[0183] The optimization unit is used to construct a multi-channel Wiener filter optimization objective function containing noise reduction terms and spatial constraint terms based on the smoothing gain coefficient, the target sound source identifier, and the microphone array signal, and solve for the optimal weight vector.

[0184] The component acquisition unit is used to perform a linear filtering operation on the microphone array signal using the optimal weight vector to obtain a spatial enhancement component, and to acquire a reference channel signal in the microphone array as a background component.

[0185] The fusion unit is used to perform a weighted synthesis operation based on the smoothing gain coefficient, spatial enhancement component, and background component to obtain the final enhanced audio signal and output it.

[0186] In this preferred embodiment, the target sound source probability value is compared with a preset high and low threshold and the duration is determined to obtain the target sound source identifier. Hysteresis logic is introduced to prevent misjudgment caused by short-term fluctuations in probability. The smoothing gain coefficient is calculated by combining a smoothing factor to control the slope of gain change and avoid sudden changes in auditory perception when switching sound sources. A multi-channel Wiener filter containing noise reduction and spatial constraint terms is constructed to optimize the objective function and solve for the optimal weight vector, which ensures the preservation of spatial cues while enhancing speech. The microphone array signal is filtered using the optimal weight vector to obtain the spatial enhancement component, and the reference channel signal is obtained as the background component, separating the enhanced signal from the ambient noise. Finally, a weighted synthesis operation is performed based on the smoothing gain coefficient, the spatial enhancement component, and the background component. The slight preservation of non-target background sound prevents the loss of spatial sense caused by complete silence. The final output is an enhanced audio signal that is both clear and loud and has a natural sense of direction, effectively reducing the user's cognitive load and maintaining a long-term auditory attention loop.

[0187] In summary, compared with existing technologies, the embodiments of this application have the following beneficial effects: By simultaneously acquiring raw EEG signals, microphone array signals, inertial measurement data, and the audio driving signal from the previous historical frame through a hearing aid, a foundation for multimodal data perception is constructed; blind source separation of the microphone array signals yields independent candidate sound source streams, enabling the deconstruction of the mixed sound field without prior sound source location information, providing independent candidate objects for attention decoding; simultaneously, based on the inertial measurement data and the audio driving signal from the previous historical frame, motion artifact cancellation and audio interference cancellation are sequentially performed on the raw EEG signals. The use of the audio driving signal from the previous historical frame as a reference is based on the physical delay in sound wave propagation from the receiver to the ear canal electrodes and the computational limitations of digital signal processing. The time-consuming causal nature of the system allows it to accurately predict and cancel electromagnetic and micro-sound interference generated at the current moment using known "past" output signals, thus ensuring the feasibility of real-time noise reduction. Combined with inertial measurement data, it effectively cuts off the annihilation effect of receiver electromagnetic / mechanical interference and user head movements on weak EEG signals, improving the signal-to-noise ratio and usability of EEG signals. Furthermore, it performs neural decoding matching analysis based on independent candidate sound source streams and purified EEG signals to obtain the target sound source probability value, enabling active sound selection based on the user's true auditory intent. Finally, it performs spatial enhancement beamforming processing based on the target sound source probability value and microphone array signals, outputting an enhanced audio signal that enhances the target speech while preserving spatial cues of background sound. These steps work synergistically to form a closed-loop control of "perception-decoding-enhancement," solving the technical challenges of signal failure due to physical interference and loss of auditory spatial sense caused by traditional hard-switching algorithms when introducing EEG control in existing hearing aids. This achieves a clear and naturally spatial intelligent hearing aid experience.

[0188] Example 3:

[0189] This invention also provides a computer program product, including a computer program or instructions, capable of running on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to execute any of the hearing aid signal processing methods based on electroencephalogram (EEG) detection of this invention.

[0190] Example 4:

[0191] This invention also provides a computer-readable storage medium storing at least one executable instruction that, when executed on a hearing aid signal processing system based on electroencephalogram (EEG) detection, causes the EEG-based hearing aid signal processing system to perform one of the EEG-based hearing aid signal processing methods described in any of the above method embodiments.

[0192] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. Similarly, for the purpose of simplification and aiding understanding of one or more aspects of the invention, in the above description of exemplary embodiments of this application, various features of the embodiments are sometimes grouped together in a single embodiment, figure, or description thereof. The claims, which follow the detailed description, are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0193] Those skilled in the art will understand that the modules in the system of the embodiments can be adaptively changed and placed in one or more systems different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components, except that at least some of such features and / or processes or units are mutually exclusive.

Claims

1. A method for processing hearing aid signals based on electroencephalogram (EEG) detection, characterized in that, include: The hearing aid simultaneously collects the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal from the previous historical frame. Blind source separation is performed on the microphone array signal to obtain independent candidate sound source streams. Based on the inertial measurement data and the audio driving signal of the previous historical frame, motion artifact cancellation processing and audio interference cancellation processing are performed on the original EEG signal to obtain the EEG signal. Based on the independent candidate sound source streams and EEG signals, neural decoding matching analysis is performed to obtain the target sound source probability value. Based on the target sound source probability value and the microphone array signal, spatial enhancement beamforming processing is performed to obtain an enhanced audio signal and output it. The step of performing blind source separation on the microphone array signal to obtain independent candidate sound source streams includes: Based on the microphone array signal, the statistical domain feature matrix is ​​obtained by calculating the received signal correlation matrix and the noise correlation matrix. Based on the statistical domain feature matrix, the acoustic steering vector of each independent sound source is identified by the generalized eigenvalue decomposition algorithm, and the spatial feature vector of the sound source is obtained. Based on the spatial feature vector of the sound source, a mutually exclusive spatial filter weight vector that satisfies the linear constraints of unity gain and null trap is constructed to obtain the spatial filter parameters; Based on the spatial filtering parameters, a linear filtering operation is performed on the microphone array signal to obtain each independent candidate sound source stream corresponding to potential sound sources in different directions; The step of performing spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and outputting it includes: The target sound source probability value is compared with a preset high and low threshold and the duration is determined to obtain the target sound source identifier. Based on the target sound source identifier, a smoothing gain coefficient is calculated in combination with a preset smoothing factor. Based on the smoothing gain coefficient, the target sound source identifier, and the microphone array signal, a multi-channel Wiener filter optimization objective function containing noise reduction and spatial constraint terms is constructed, and the optimal weight vector is obtained by solving the problem. The optimal weight vector is used to perform a linear filtering operation on the microphone array signal to obtain a spatial enhancement component, and the reference channel signal in the microphone array is obtained as the background component. Based on the smoothing gain coefficient, spatial enhancement component, and background component, a weighted synthesis operation is performed to obtain the final enhanced audio signal and output it. The process of performing motion artifact cancellation and audio interference cancellation on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal includes: Based on the inertial measurement data, a first adaptive filter is constructed using a recursive least squares algorithm to estimate the motion artifact noise estimation signal related to the inertial measurement data and filter it out from the original EEG signal to obtain an intermediate EEG signal. Based on the audio driving signal of the previous historical frame, a second adaptive filter is constructed using the normalized least mean square algorithm to estimate the audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and then filtered out in the intermediate state EEG signal to obtain the EEG signal.

2. The hearing aid signal processing method based on electroencephalogram (EEG) detection as described in claim 1, characterized in that, The step of constructing a first adaptive filter based on the inertial measurement data using a recursive least squares algorithm, estimating the motion artifact noise estimation signal related to the inertial measurement data, and filtering it out from the original EEG signal to obtain an intermediate-state EEG signal includes: Based on the duration of the original EEG signal, the discrete time step is initialized, and the inverse correlation matrix and filter weight vector of the recursive least squares algorithm are initialized. Based on the inertial measurement data and the original EEG signal, a recursive least squares iterative process is performed until all discrete time steps are traversed, and the final intermediate EEG signal is output. In each iteration, the triaxial acceleration and triaxial angular velocity data prior to the current time step are extracted from the inertial measurement data and a first reference vector is constructed. The gain vector is calculated using the current inverse correlation matrix, a preset forgetting factor, and the first reference vector. Combined with the current filter weight vector and the first reference vector, the motion artifact noise estimation signal for the current time step is estimated. The motion artifact noise estimation signal is subtracted from the original EEG signal for the current time step to obtain the intermediate-state EEG signal for the current time step, which is used as the prior error for this round. The current filter weight vector and inverse correlation matrix are updated based on the gain vector, the conjugate signal of the prior error, and the forgetting factor.

3. The hearing aid signal processing method based on electroencephalogram (EEG) detection as described in claim 1, characterized in that, The step involves constructing a second adaptive filter based on the audio driving signal of the previous historical frame using a normalized least mean square algorithm, estimating the audio interference noise estimate signal related to the audio driving signal of the previous historical frame, and filtering it out from the intermediate-state EEG signal to obtain the EEG signal, including: Based on the duration of the intermediate-state EEG signal, the discrete time step is initialized, and the filter weight vector of the normalized least mean square algorithm is initialized. Based on the audio driving signal of the previous historical frame and the intermediate EEG signal, normalized least mean square iteration processing is performed until all discrete time steps are traversed, and the final EEG signal is output. In each iteration, audio data prior to the current time step is extracted from the previous frame's audio driving signal and a second reference vector is constructed. The second reference vector is then convolved with the current filter weight vector to estimate the audio interference noise signal at the current time step. The estimated audio interference noise signal is then filtered out from the intermediate EEG signal at the current time step to obtain the EEG signal at the current time step. The signal energy of the second reference vector is calculated, and the filter weight vector for the next round is updated using the signal energy, the EEG signal at the current time step, and the second reference vector, according to the normalized least mean square criterion.

4. The hearing aid signal processing method based on electroencephalogram (EEG) detection as described in claim 1, characterized in that, The step of obtaining the target sound source probability value by performing neural decoding matching analysis based on the independent candidate sound source streams and EEG signals includes: The independent candidate sound source streams are subjected to gamma-pass filtering and the amplitude envelope after power-law compression is extracted to obtain auditory envelope features that simulate the characteristics of the human cochlea. The EEG signal was bandpass filtered to extract neural oscillation features covering the neural entrainment region and the attention control region; The auditory envelope features and the neural oscillation features are input into a pre-trained convolutional neural network model, and the matching probability is calculated within a preset time window. The output is a target sound source probability value that indicates the degree to which the user pays attention to a specific speaker.

5. A hearing aid signal processing system based on electroencephalogram (EEG) detection, characterized in that, include: Data acquisition module, preprocessing module, and result output module; The data acquisition module is used to synchronously collect the user's original EEG signals, microphone array signals, inertial measurement data, and the audio drive signal of the previous historical frame through the hearing aid. The pre-processing module is used to perform blind source separation on the microphone array signal to obtain independent candidate sound source streams, and to perform motion artifact cancellation processing and audio interference cancellation processing on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal. The result output module is used to perform neural decoding matching analysis based on the independent candidate sound source stream and EEG signal to obtain the target sound source probability value, and to perform spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and output it. The step of performing blind source separation on the microphone array signal to obtain independent candidate sound source streams includes: Based on the microphone array signal, the statistical domain feature matrix is ​​obtained by calculating the received signal correlation matrix and the noise correlation matrix. Based on the statistical domain feature matrix, the acoustic steering vector of each independent sound source is identified by the generalized eigenvalue decomposition algorithm, and the spatial feature vector of the sound source is obtained. Based on the spatial feature vector of the sound source, a mutually exclusive spatial filter weight vector that satisfies the linear constraints of unity gain and null trap is constructed to obtain the spatial filter parameters; Based on the spatial filtering parameters, a linear filtering operation is performed on the microphone array signal to obtain each independent candidate sound source stream corresponding to potential sound sources in different directions; The step of performing spatial enhancement beamforming processing based on the target sound source probability value and the microphone array signal to obtain an enhanced audio signal and outputting it includes: The target sound source probability value is compared with a preset high and low threshold and the duration is determined to obtain the target sound source identifier. Based on the target sound source identifier, a smoothing gain coefficient is calculated in combination with a preset smoothing factor. Based on the smoothing gain coefficient, the target sound source identifier, and the microphone array signal, a multi-channel Wiener filter optimization objective function containing noise reduction and spatial constraint terms is constructed, and the optimal weight vector is obtained by solving the problem. The optimal weight vector is used to perform a linear filtering operation on the microphone array signal to obtain a spatial enhancement component, and the reference channel signal in the microphone array is obtained as the background component. Based on the smoothing gain coefficient, spatial enhancement component, and background component, a weighted synthesis operation is performed to obtain the final enhanced audio signal and output it. The process of performing motion artifact cancellation and audio interference cancellation on the original EEG signal based on the inertial measurement data and the audio driving signal of the previous historical frame to obtain the EEG signal includes: Based on the inertial measurement data, a first adaptive filter is constructed using a recursive least squares algorithm to estimate the motion artifact noise estimation signal related to the inertial measurement data and filter it out from the original EEG signal to obtain an intermediate EEG signal. Based on the audio driving signal of the previous historical frame, a second adaptive filter is constructed using the normalized least mean square algorithm to estimate the audio interference noise estimation signal related to the audio driving signal of the previous historical frame, and then filtered out in the intermediate state EEG signal to obtain the EEG signal.

6. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed, they implement a hearing aid signal processing method based on electroencephalogram (EEG) detection as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a hearing aid signal processing method based on electroencephalogram (EEG) detection as described in any one of claims 1-4.

Images