Portable directional hearing aid device based on nerve guidance and signal processing method

Abstract

The invention discloses portable directional hearing aid equipment based on nerve guidance and a signal processing method, and the equipment comprises an electroencephalogram signal collection module which is configured to collect an electroencephalogram signal, related to the auditory attention direction, of a user; an acoustic acquisition system configured to acquire a sound signal corresponding to a sound stimulus in an environment around a user; the calculation module is coupled to the electroencephalogram probe and the microphone array and used for processing the collected electroencephalogram signals and sound signals; and the audio output device is coupled to the calculation module and is configured to play the processed sound to a user. Wherein the calculation module decodes spatial-temporal characteristics of electroencephalogram signals through a convolutional neural network so as to predict the attention direction of a user; the sound signals are processed through a weighted beam forming algorithm, and sound output in the direction in which the user is interested is generated; besides, the sound output intensity is dynamically adjusted according to the hearing loss condition of the user, so that the directional sound amplification capability of the hearing aid equipment in a complex multi-sound-source scene is remarkably enhanced, and the hearing experience and social contact convenience of the user are improved.

Description

Technical Field

[0001] The present application relates to the fields of signal processing and hearing assistance technology, and in particular to a portable directional hearing aid device based on neural guidance and a signal processing method. Background Art

[0002] In an environment with multiple sound sources, humans can ignore interfering sounds in other directions by focusing on sounds in a certain area. However, hearing-impaired people find it difficult to focus on sounds in a specific spatial direction in a noisy environment. In a complex sound field environment, patients may want to hear the target sound coming from the left or right front. However, traditional hearing aids can only compensate for hearing loss through simple sound amplification. They lack the ability to accurately perceive the direction of the user's attention and directional enhancement, and cannot meet the user's needs for directional listening in complex multi-sound source scenarios. Therefore, there is an urgent need to provide hearing-impaired people with a directional hearing aid device that enables them to focus on the direction of sounds of interest in complex environments, thereby improving their daily lives and social activities.

[0003] EEG signal acquisition equipment records brain wave signals related to the direction of auditory attention through electrodes placed at specific locations on the user's head. By decoding the EEG signals using signal processing technology, the direction of the user's auditory attention can be identified. However, traditional EEG acquisition equipment mostly uses EEG caps or multi-channel EEG probes (such as 16, 32 or 64 channels). Although the signal accuracy is high, the equipment is large and complicated to operate, making it unsuitable for daily use. Therefore, it is necessary to optimize the EEG signal acquisition and processing methods, locate the scalp area most related to auditory attention, and develop portable equipment by reducing the number of EEG probe channels to achieve daily application. Summary of the Invention

[0004] In response to the shortcomings of current technology, the purpose of the present invention is to propose a portable directional hearing aid device and signal processing method based on neural guidance. Using this device, users can collect EEG signals through a probe and guide the device to strengthen auditory attention to the direction of sound.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A portable directional hearing aid device based on neural guidance, comprising an EEG signal acquisition module, an acoustic acquisition system, a computing module, and an audio output device;

[0007] The EEG signal acquisition module uses an 8-channel EEG probe that accurately covers the prefrontal cortex and superior temporal gyrus area, capturing neural activities and event-related potentials related to auditory attention in real time;

[0008] The acoustic collection system uses a neck-worn annular microphone array for multi-directional sound collection;

[0009] The computing module integrates a neural decoding engine and a digital signal processor. The neural decoding part is based on a convolutional neural network and extracts the user's auditory attention direction through time-frequency domain joint analysis and decoding of EEG signals. The digital signal processor synthesizes the sound of the user's auditory attention direction and amplifies the synthesized sound according to the user's hearing loss.

[0010] The audio output device obtains data from the EEG probe through a wired connection, and exchanges data with the computing module wirelessly, transmitting the data collected by the EEG probe to the computing module, while obtaining the data transmitted by the computing module and playing the processed sound to the user.

[0011] Preferably, the spatial resolution of the EEG probe reaches the international 10-20 system standard, effectively analyzing the user's spatial attention direction to different sound sources.

[0012] Preferably, the annular microphone array is composed of N cardioid directional microphones, which are evenly distributed on the circumference and have an angular interval of 2π / N between each other; the directional microphones have strong forward sensitivity and weak back sensitivity, and can effectively suppress the sound from the back.

[0013] Preferably, the audio output device uses a Bluetooth headset. Using a Bluetooth headset does not require wiring and can achieve wireless transmission of sound signals from the signal processing module to the headset, thereby improving the portability and aesthetics of the system and meeting the hearing aid needs of daily wear.

[0014] The signal processing method of the portable directional hearing aid device based on nerve guidance comprises the following steps:

[0015] Step 1: Sounds are played in different directions, and participants are asked to focus on a specific direction. An 8-channel EEG probe is used to collect EEG signals. To extract information about the spatial location of the sound contained in the cerebral cortex, the collected signals are bandpass filtered. A short-time Fourier transform is then used to generate a time-frequency diagram containing channel-time-frequency information, which is then labeled according to the spatial direction of the sound.

[0016] Step 2: Use a convolutional neural network to learn the mapping relationship between EEG signals and directional labels, obtaining a directional decoder that can extract the spatiotemporal features of EEG signals. Step 1 is used to collect EEG signal time-frequency data from multiple participants to construct a training dataset for the convolutional neural network. The convolutional neural network uses multiple convolutional and pooling layers to extract multi-level feature representations, followed by a fully connected layer for directional classification.

[0017] Step 3: The EEG signals collected in real time by the 8-channel EEG probe are input into the directional decoder, which outputs correlation scores for different directions. The directional decoder selects the direction with the highest directional response value as the inference of the user's current attention direction.

[0018] Step 4: Use the ring microphone array mounted on the neck to collect ambient sound and transmit it wirelessly to the computing module;

[0019] Step 5: The output of the directional decoder obtained in step 3 is the user's attention direction, which is recorded as θ u , the pointing direction of the i-th microphone in the microphone array is recorded as θ i ; By calculating |θ u -θ i | Find the microphone closest to the user's attention direction, record it as the main microphone, and the direction is θ m , the angle between it and the user's attention direction is θ=|θ u -θ m |; The angle between the adjacent second-closest microphone and the user's attention direction is 2π / N-θ; The portable directional hearing aid device uses a weighted beamforming algorithm to synthesize the sound output S in the direction of the user's attention:

[0020]

[0021] S m and S n The sound signals collected by the main microphone and the secondary close microphone respectively;

[0022] Step 6: Based on the user's hearing loss, the computing module amplifies the sound output S and plays it to the user through headphones.

[0023] Preferably, the bandpass filtering adopts 0.1 to 30 Hz. In human EEG signals, the EEG frequency range closely related to auditory attention is 0.1 to 30 Hz. While retaining the auditory attention characteristics, the high-frequency noise caused by myoelectric interference such as blinking and chewing and the low-frequency drift caused by non-neural factors such as skin resistance changes and sweat are removed, which helps the convolutional neural network to extract real and effective neural features.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] 1. Using an 8-channel EEG probe greatly reduces the size and complexity of the device, and enhances portability and wearing comfort.

[0026] 2. The neck-worn ring microphone array uses a cardioid directional microphone to effectively suppress noise from non-directional directions. The ring layout is suitable for scenarios with multi-angle sound sources, improving practicality in noisy environments.

[0027] 3. Use convolutional neural networks to classify the direction of EEG signals and achieve high-accuracy decoding of the user's attention direction while maintaining a low number of channels.

[0028] 4. Use weighted beamforming algorithms to achieve auditory enhancement, directionally amplifying sounds in the direction of the user's auditory attention while suppressing noise in other directions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1a and Figure 1b They are respectively structural schematic diagrams of the EEG signal acquisition module and the audio output device of the present invention.

[0030] Figure 2 Schematic diagram of the acoustic acquisition system and intelligent processing unit structure.

[0031] In the picture, 1. EEG probe, 2. Bluetooth headset, 3. microphone, 4. computing module, 5. neckband.

[0032] Figure 3 This is a flow chart of the signal processing method of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in further detail below with reference to the accompanying drawings:

[0034] A portable directional hearing aid device based on nerve guidance includes an electroencephalogram (EEG) signal acquisition module, an acoustic acquisition system, a computing module, and an audio output device.

[0035] As shown in Figure 1, the EEG signal acquisition module utilizes an 8-channel EEG probe that complies with the international 10-20 system standard. This 8-channel probe features an 8-channel dry electrode array1, covering the prefrontal cortex and superior temporal gyrus. Its spatial resolution reaches ±2cm, enabling real-time capture of cortical auditory attention-related neural activity. The electrode array incorporates an integrated electromagnetic shielding layer, with impedance controlled below 5kΩ. With a sampling frequency of 1000Hz, it accurately extracts the N1 / P2 components of event-related potentials and steady-state auditory evoked potentials, enabling analysis of the user's spatially directed attention to a target sound source.

[0036] The audio output device uses a Bluetooth headset 2, which obtains data through a wired connection with the EEG probe and exchanges data with the computing module through a wireless connection. It transmits the data collected by the EEG probe to the computing module, and at the same time obtains the data transmitted by the computing module and plays the processed sound to the user.

[0037] like Figure 2As shown, the acoustic collection system uses a neck-worn annular microphone array consisting of eight cardioid MEMS microphones3 with an array diameter of 80mm, each microphone evenly distributed at 60°. Each microphone has a forward sensitivity of -26dB±1.5dB at 1kHz and a backscatter rejection ratio of 15dB, forming a directional beam with a frequency range of 200Hz-8kHz.

[0038] Computing module 4 integrates a neural decoding engine and a digital signal processor. The neural decoding component, based on a convolutional neural network, decodes EEG signals through a joint analysis in the time and frequency domains to extract the direction of the user's auditory attention. The digital signal processor synthesizes the sound indicating the user's auditory attention direction and amplifies the synthesized sound based on the user's hearing loss. The microphone array is connected to computing module 4 via a neckband 5.

[0039] like Figure 3 As shown, a signal processing method for a portable directional hearing aid device based on nerve guidance includes the following steps:

[0040] Step 1: This example uses speech and EEG data. Eight channels of EEG data were recorded from 22 subjects using the device of the present invention at a sampling rate of 512 Hz. The EEG probes were positioned over the prefrontal lobe and superior temporal gyrus regions of the brain, following the international 10-20 system.

[0041] The subjects were placed at the center of a circle of eight evenly spaced speakers, each spaced 45° apart. During the experiment, the subjects faced speaker number 1. A randomly selected speaker played for 30 seconds, and the subjects were instructed to focus on the sound in that direction. The direction of the speaker and the subject's EEG signal at that moment were recorded. The experiment then repeated this process with the remaining speakers until all eight speakers had played. A total of 22 subjects completed 176 trials, resulting in a cumulative EEG recording of 88 minutes.

[0042] The EEG signal is downsampled to 40Hz and bandpass filtered from 0.1 to 30Hz to extract the information it contains about the spatial location of the sound. The processed signal is then subjected to a short-time Fourier transform to generate a time-frequency diagram containing channel-time-frequency information, which is then labeled according to the spatial direction of the sound.

[0043] Step 2: Use a convolutional neural network to learn the mapping between EEG signals and directional labels, generating a directional decoder capable of extracting the spatiotemporal features of EEG signals. The EEG frequency maps and directional labels from the 22 subjects obtained in Step 1 form a training dataset for the convolutional neural network. The convolutional neural network uses multiple convolutional and pooling layers to extract multi-level feature representations, followed by a fully connected layer for directional classification.

[0044] The data from 176 trials were divided into training, validation, and test sets in a 6:1:1 ratio. The cross-entropy loss function was used to calculate the error between the directional labels predicted by the convolutional neural network and the true labels. Based on the gradient of the loss function, the backpropagation algorithm was used to update the weights and biases in the convolutional neural network model, gradually reducing the error in the convolutional neural network model's predictions. The gradient descent optimization algorithm, Adam, was used for iterative optimization, allowing the convolutional neural network model to gradually learn the mapping relationship between EEG signals and directional labels with each iteration. The hyperparameters of the nonlinear model (such as the number of hidden units, learning rate, input latitude, and number of hidden layers) were optimized to ensure rapid convergence and optimal prediction performance during training. After achieving optimal performance on the validation set, the convolutional neural network model was evaluated on the test set to verify its accuracy and generalization ability in classifying directional labels.

[0045] Step 3: The EEG signals collected in real time by the 8-channel EEG probe are fed into a directional decoder. The convolutional neural network model outputs correlation scores for different directions. The directional decoder selects the direction with the highest directional response value as the inference of the user's current attention direction.

[0046] According to the methods of steps 1 and 2, 64-channel, 32-channel, and 16-channel EEG probes were used to collect EEG signals and train directional decoders. Experimental results show that compared with the attention direction detection accuracy of the 64-channel system, the accuracy of the 32-channel system decreased by 2.1%, the 16-channel system decreased by 4.2%, and the 8-channel system decreased by 7.1%. The 8-channel system is close to the multi-channel system in performance, while retaining the core ability to detect the direction of auditory attention. Compared with traditional EEG caps and 16-channel EEG probes, the 8-channel EEG probe can greatly reduce the number of sensors, realize the miniaturization and portability of the equipment, and significantly enhance the wearing comfort of the user.

[0047] Step 4: A circular microphone array mounted on the neck collects ambient sound and transmits it wirelessly to the computing module. The array consists of eight cardioid directional microphones evenly distributed around the circumference, with an angular spacing of 2π / 8. Directional microphones have strong forward sensitivity and weak back sensitivity, effectively suppressing sounds from behind. A larger number of microphones in the array enhances the directional sound, but also reduces wearing comfort.

[0048] Step 5: The output of the directional decoder obtained in step 3 (the direction of user attention) is recorded as θ u , the pointing direction of the i-th microphone in the microphone array is recorded as θ i By calculating |θ u -θ i| Find the microphone closest to the user's attention direction, record it as the main microphone, and the direction is θ m , the angle between it and the user's attention direction is θ=|θ u -θ m The angle between the adjacent second-closest microphone and the user's attention direction is 2π / 8-θ. The portable directional hearing aid uses a weighted beamforming algorithm to synthesize the sound output S in the direction of the user's attention:

[0049]

[0050] S m and S n The sound signals collected by the main microphone and the secondary close microphone respectively.

[0051] Step 6: Based on the user's hearing loss, the computing module amplifies the sound output S and plays it to the user through headphones.

Claims

1. A portable directional hearing aid device based on neural guidance, characterized by: It includes an EEG signal acquisition module, an acoustic acquisition system, a computing module and an audio output device; The EEG signal acquisition module uses an 8-channel EEG probe that accurately covers the prefrontal cortex and superior temporal gyrus area, capturing neural activities and event-related potentials related to auditory attention in real time; The acoustic collection system uses a neck-worn annular microphone array for multi-directional sound collection; The computing module integrates a neural decoding engine and a digital signal processor. The neural decoding part is based on a convolutional neural network, which extracts the user's auditory attention direction by jointly analyzing and decoding EEG signals in the time and frequency domains. The digital signal processor synthesizes the sound in the direction of the user's auditory attention and amplifies the synthesized sound according to the user's hearing loss; The audio output device obtains data from the EEG probe through a wired connection, and exchanges data with the computing module wirelessly, transmitting the data collected by the EEG probe to the computing module, while obtaining the data transmitted by the computing module and playing the processed sound to the user.

2. The portable directional hearing aid device based on nerve guidance according to claim 1, characterized in that: The spatial resolution of the EEG probe meets the international 10-20 system standard, effectively analyzing the user's spatial attention direction to different sound sources.

3. The portable directional hearing aid device based on nerve guidance according to claim 1, characterized in that: The annular microphone array consists of N cardioid directional microphones, which are evenly distributed on the circumference with an angular interval of 2π / N between each other. The directional microphones have strong forward sensitivity and weak back sensitivity, and can effectively suppress the sound from the back.

4. The portable directional hearing aid device based on neural guidance according to claim 1, characterized in that: The audio output device uses a Bluetooth headset.

5. The signal processing method for a portable directional hearing aid device based on neural guidance according to any one of claims 1 to 4, characterized in that: The following steps are involved: Step 1: Sounds are played in different directions, and participants are asked to focus on a specific direction. An 8-channel EEG probe is used to collect EEG signals. To extract information about the spatial location of the sound contained in the cerebral cortex, the collected signals are bandpass filtered. A short-time Fourier transform is then used to generate a time-frequency diagram containing channel-time-frequency information, which is then labeled according to the spatial direction of the sound. Step 2: Use a convolutional neural network to learn the mapping relationship between EEG signals and directional labels, obtaining a directional decoder that can extract the spatiotemporal features of EEG signals. Step 1 is used to collect EEG signal time-frequency data from multiple participants to construct a training dataset for the convolutional neural network. The convolutional neural network uses multiple convolutional and pooling layers to extract multi-level feature representations, followed by a fully connected layer for directional classification. Step 3: The EEG signals collected in real time by the 8-channel EEG probe are input into the directional decoder, which outputs correlation scores for different directions. The directional decoder selects the direction with the highest directional response value as the inference of the user's current attention direction. Step 4: Use the ring microphone array mounted on the neck to collect ambient sound and transmit it wirelessly to the computing module; Step 5: The output of the directional decoder obtained in step 3 is the user's attention direction, which is recorded as θ u , the pointing direction of the i-th microphone in the microphone array is recorded as θ i ; By calculating |θ u -θ i | Find the microphone closest to the user's attention direction, record it as the main microphone, and the direction is θ m , the angle between it and the user's attention direction is θ=|θ u -θ m |; The angle between the adjacent second-closest microphone and the user's attention direction is 2π / N-θ; The portable directional hearing aid device uses a weighted beamforming algorithm to synthesize the sound output S in the direction of the user's attention: S m and S n The sound signals collected by the main microphone and the secondary close microphone respectively; Step 6: Based on the user's hearing loss, the computing module amplifies the sound output S and plays it to the user through headphones.

6. The signal processing method according to claim 5, wherein: The band-pass filtering adopts 0.1 to 30 Hz.

Images