Hearing aid auxiliary system based on artificial intelligence

Abstract

The invention discloses a hearing aid auxiliary system based on artificial intelligence, which belongs to the technical field of hearing aids and comprises a bone conduction hearing aid, a pickup module and a personal terminal. The pickup module is mounted on the bone conduction hearing aid and used for collecting ambient sound and extracting clear voice information in the ambient sound; and the personal terminal is used for controlling the bone conduction hearing aid to carry out an auxiliary hearing test, and is used for controlling the pickup module. According to the hearing aid auxiliary system, voice processing, artificial intelligence and other technologies are combined, intelligent analysis and adaptation can be carried out on the hearing requirement of the user and the environment, the personalized adaptive capacity and effect of the hearing aid are improved, the user can obtain better hearing experience, meanwhile, the hearing aid auxiliary system has certain intelligent learning capacity, and the user experience is improved. The system performance can be continuously learned and optimized, and the satisfaction degree and life quality of the user are improved.

Description

Technical Field

[0001] This invention relates to the field of hearing aid technology, and more specifically to an artificial intelligence-based hearing assistance system. Background Technology

[0002] Hearing loss can severely impact the physical and mental health of hearing-impaired individuals, and wearing hearing aids is currently the most effective way for them to improve their hearing. Existing hearing aid technology has played a significant role in improving hearing outcomes and quality of life for people with hearing loss. However, due to the diverse auditory needs and living environments of people with hearing loss, conventional hearing aid technology struggles to meet their individualized needs and address the challenges of complex environments.

[0003] Therefore, this invention proposes an artificial intelligence-based hearing aid system to improve the adaptability, intelligence, and user experience of hearing aids. Summary of the Invention

[0004] The technical problem to be solved by this invention is: how to improve the adaptability, intelligence and user experience of hearing aids. It provides an artificial intelligence-based hearing aid system that uses the OverIVA algorithm and the least squares GAN model to process the collected audio information to obtain clear speech information. At the same time, it can also perform hearing tests on users and formulate sound compensation schemes based on the test results.

[0005] The present invention solves the above-mentioned technical problems through the following technical solution: The present invention includes a bone conduction hearing aid, a pickup module, and a personal terminal; the pickup module is installed on the bone conduction hearing aid and is used to collect ambient sounds and extract clear speech information from the ambient sounds; the personal terminal is used to control the bone conduction hearing aid to perform assisted hearing tests and to control the pickup module.

[0006] The sound pickup module includes a sound pickup device, a noise reduction unit, a voice extraction unit, an information enhancement unit, and a direction determination unit. The sound pickup device is used to collect ambient sound. The noise reduction unit is used to process the ambient sound to obtain a processed sound signal. The voice extraction unit is used to extract speech information from the processed sound signal. The information enhancement unit is used to enhance the speech information to obtain clear speech information with a signal-to-noise ratio improvement of ≥15dB and a speech intelligibility score of ≥3.5 on a 5-point scale, and transmits the clear speech information to the personal terminal. The direction determination unit is used to determine the direction of sound emission based on an adaptive beamforming method.

[0007] The personal terminal includes an ambient sound detection unit, a hearing test unit, a control unit, a conversion unit, and an interaction unit. The ambient sound detection unit receives ambient sounds and determines whether the current environment meets the testing requirements. The hearing test unit performs a hearing test on the wearer and generates a hearing loss level report, which is a report of the user's hearing threshold. The control unit generates a compensation scheme based on the hearing loss level report and performs sound compensation on the bone conduction hearing aid based on the compensation scheme and the sound emission direction. The control unit also controls the sound pickup direction of the pickup module. The conversion unit receives the clear speech information and converts it into text information. The interaction unit provides human-computer interaction and displays the text information.

[0008] Furthermore, in the direction determination unit, multiple sound signals from the microphone array of the pickup device are collected, the time delay difference and relative intensity difference of the signals are calculated, and then the multiple received sound signals are dynamically weighted using the variable step size normalized least mean square algorithm to improve the signal-to-noise ratio of the wavefront. Finally, the weighted signals are merged to estimate the direction of sound emission.

[0009] Furthermore, the voice extraction unit includes a receiving device, a dereverberation device, and a second DSP chip; the receiving device is used to receive the processed sound signal; the dereverberation device is used to perform dereverberation processing on the processed sound signal to obtain a dereverberated sound signal; the second DSP chip is used to perform blind source separation on the dereverberated sound signal to obtain the speech information.

[0010] Furthermore, in the dereverberation device, the dereverberation process involves performing a short-time Fourier transform on the processed sound signal to convert the time-domain signal into a frequency-domain signal; then, a normalized least mean square adaptive algorithm is applied to perform multi-channel dereverberation, effectively eliminating environmental echoes and reverberation effects to obtain a dereverberated sound signal.

[0011] Furthermore, in the second DSP chip, the specific process of blind source separation is as follows: based on the OverIVA blind source separation algorithm, the dereverberant sound signal after dereverberation processing is decomposed, and by maximizing the independence between signal sources, the target speech component is separated and the background noise is suppressed to obtain the purified target speech signal, that is, the speech information is obtained.

[0012] Furthermore, the noise reduction unit includes a first low-pass filter, a high-pass filter, a first DSP chip, and a second low-pass filter. The first low-pass filter is used to filter out high-frequency noise in the ambient sound and input the high-frequency phase signal to the first DSP chip. The high-pass filter is used to filter out low-frequency noise in the ambient sound and input the low-frequency phase signal to the first DSP chip. The first DSP chip is used to perform phase superposition of the high-frequency phase signal and the low-frequency phase signal to obtain a superimposed signal. The second low-pass filter is used to filter out high-frequency residual noise in the superimposed signal to obtain the processed sound signal.

[0013] Furthermore, in the information enhancement unit, the speech information is enhanced based on the least squares GAN model. The specific enhancement process is as follows: the generator G is used to denoise and compensate the spectrum of the input speech information, and the discriminator D judges the generated result based on the least squares loss function. Through adversarial training between the generator and the discriminator, the generator G finally outputs clear speech information.

[0014] Furthermore, within the control unit, based on the frequency adaptive gain model generated from the hearing loss level report, the amplification curve of the bone conduction hearing aid is dynamically adjusted to obtain a compensation scheme. The specific process is as follows:

[0015] S21: Extract hearing thresholds for each core frequency within the 125Hz-8kHz band from the hearing loss level report. and average hearing threshold ;

[0016] S22: Calculate the target gain for each frequency band based on the NAL-NL2 prescription formula. Construct a frequency-adaptive gain model;

[0017] S23: Determine the angle of the sound source output by the unit based on the direction. Spatial filtering optimization is performed using the weight vector of the MVDR beamformer;

[0018] S24: Convert the target gain and weight vector of each frequency band into the amplification curve adjustment parameters of the bone conduction hearing aid, adapt them to the built-in amplification channel of the hearing aid, set the maximum gain safety threshold, and update the amplification curve.

[0019] S25: Integrate the adjusted amplification curve parameters into a complete compensation scheme and transmit it to the audio output module of the bone conduction hearing aid.

[0020] Furthermore, the frequency adaptive gain model calculates the target gain for each frequency band based on the NAL-NL2 prescription formula. , ,in For frequency Hearing threshold, This represents the average hearing threshold.

[0021] Furthermore, in the hearing test unit, the hearing test process is as follows:

[0022] Multiple test audios with different frequencies and decibels are generated and sent to the user through the bone conduction hearing aid;

[0023] The user selects whether they can hear the test audio based on the prompts from the interactive unit;

[0024] Based on the user's selection, the hearing threshold at each frequency is calculated using cubic spline interpolation to generate the hearing loss level report.

[0025] Compared with the prior art, the present invention has the following advantages: This artificial intelligence-based hearing aid system combines speech processing and artificial intelligence technologies, which can intelligently analyze and adapt to the user's hearing needs and environment, improve the personalized adaptability and effect of the hearing aid, and give the user a better auditory experience. At the same time, the hearing aid system of the present invention has a certain intelligent learning ability, which can continuously learn and optimize system performance, improve user satisfaction and quality of life. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the hearing aid system in an embodiment of the present invention. Detailed Implementation

[0027] The embodiments of the present invention are described in detail below. These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. However, the scope of protection of the present invention is not limited to the following embodiments.

[0028] Example 1

[0029] like Figure 1 As shown, this embodiment provides a technical solution: an artificial intelligence-based hearing aid system, including a bone conduction hearing aid, a pickup module, and a personal terminal; the pickup module is installed on the bone conduction hearing aid and is used to collect ambient sounds and extract clear speech information from the ambient sounds; the personal terminal is used to control the bone conduction hearing aid to perform assisted hearing tests and to control the pickup module.

[0030] In this embodiment, the sound pickup module includes a sound pickup device, a noise reduction unit, a voice extraction unit, an information enhancement unit, and a direction determination unit. The sound pickup device is used to collect ambient sound. The noise reduction unit is used to perform noise reduction processing on the ambient sound to obtain a processed sound signal. The voice extraction unit is used to extract speech information from the processed sound signal. The information enhancement unit is used to enhance the speech information to obtain clear speech information with a signal-to-noise ratio improvement of ≥15dB and a 5-point speech intelligibility score (PESQ) of ≥3.5, and transmits the clear speech information to the personal terminal. The direction determination unit is used to determine the direction of sound emission based on an adaptive beamforming method.

[0031] In this embodiment, the direction determination unit collects multiple sound signals from the microphone array of the pickup device, calculates the time delay difference and relative intensity difference of the signals, then uses an adaptive algorithm to dynamically weight the received multiple sound signals to improve the signal-to-noise ratio of the wavefront, and then performs merging processing on the weighted signals to estimate the direction of sound emission.

[0032] More specifically, the adaptive algorithm is the "Variable Step-Size Normalized Least Mean Square (VSS-NLMS) algorithm".

[0033] The core principle of the VSS-NLMS algorithm is as follows:

[0034] The VSS-NLMS algorithm is an improved version of the NLMS algorithm. Its core idea is to dynamically adjust the iteration step size based on the statistical characteristics of the signal and noise, achieving signal weighted optimization with "fast convergence and low steady-state error". Its core formula is as follows:

[0035] 1. Step size update formula

[0036]

[0037] in, : No. The adaptive step size for the next iteration (range: ); Maximum step size (set to 0.8 in this system to balance convergence speed and stability); : No. The error signal of the next iteration (the difference between the reference signal and the microphone array signal); : No. The microphone array input signal vector for the next iteration;

[0038] Regularization parameter (set to) (To avoid denominators being zero and improve algorithm robustness).

[0039] 2. Weight Update Formula

[0040]

[0041] in, : No. The next iteration's microphone array signal weighted vector (its dimension is consistent with the number of microphones; this system's microphone array contains 4 microphones, therefore...) (a 4×1 vector). The weight vector from the previous iteration; : Transpose of the input signal vector.

[0042] 3. Core Logic

[0043] When the target sound source signal just appears or the environmental noise changes abruptly, the error signal... Larger, step size Automatically increase (approach) The weight vector is adjusted rapidly to achieve rapid signal convergence.

[0044] When the signal is stable (the direction of the target sound source is fixed and the noise is stable), the error signal Decrease, step size The weight vector is automatically reduced and iterated slowly to reduce steady-state error and ensure the stability of signal weighting.

[0045] The specific implementation process of the VSS-NLMS algorithm in the direction determination unit is as follows:

[0046] 1. Preprocessing: Signal alignment and parameter initialization

[0047] A microphone array (4 microphones arranged in an equilateral triangle with a spacing of 2cm) collects ambient sound signals, resulting in 4 raw signals. Based on the results of calculating the time delay difference (TDOA) and relative intensity difference (IID) of the signals, time alignment is performed on the four signals (to compensate for the propagation delay from different microphones to the sound source), resulting in the aligned signal vector. ( (for discrete time points); initialize the weight vector. (Uniformly weighted), set maximum step size Regularization parameters .

[0048] 2. Adaptive weighted iteration (core step)

[0049] Reference signal generation: The signal with the highest intensity among the time-aligned signals is used as the reference signal. (Preliminary determination indicates the signal is from the direction of the target sound source);

[0050] Error calculation: Calculate the error signal (Reflects the difference between the current weighted signal and the reference signal);

[0051] Step size adjustment: Calculate the current step size according to the step size update formula. Dynamically adapt to changes in signal (such as when the sound source moves). Increase, noise stabilizes (reduce)

[0052] Weight update: Adjusted via weight update formula This increases the signal weight in the direction of the target sound source and decreases the signal weight in the direction of non-target (noise direction).

[0053] 3. Signal combining and direction estimation

[0054] After iterative convergence (error) (If the number of iterations is ≤50), the weighted signals are linearly combined: The merged signal The wavefront signal-to-noise ratio is improved by ≥12dB compared to the original signal;

[0055] Based on the merged signal The time delay difference and relative intensity difference between the original four signals are combined with the direction estimation model of the adaptive beamforming method (such as azimuth angle calculation based on phase delay) to finally output the azimuth angle of the sound source direction. (scope: ), accuracy error ≤ ±5°.

[0056] In this invention, the VSS-NLMS algorithm is not only compatible with the hardware resources and real-time requirements of the system, but also improves the accuracy of sound source direction estimation through dynamic weighted optimization, thus fully meeting the functional requirements of the direction judgment unit.

[0057] In this embodiment, the noise reduction unit includes a first low-pass filter, a high-pass filter, a first DSP chip, and a second low-pass filter. The first low-pass filter is used to filter out high-frequency noise in the ambient sound and input the high-frequency phase signal to the first DSP chip. The high-pass filter is used to filter out low-frequency noise in the ambient sound and input the low-frequency phase signal to the first DSP chip. The first DSP chip is used to perform phase superposition of the high-frequency phase signal and the low-frequency phase signal to obtain a superimposed signal. The second low-pass filter is used to filter out high-frequency residual noise in the superimposed signal to obtain the processed sound signal.

[0058] In this embodiment, the voice extraction unit includes a receiving device, a dereverberation device, and a second DSP chip; the receiving device is used to receive the processed sound signal; the dereverberation device is used to perform dereverberation processing on the processed sound signal to obtain a dereverberated sound signal; the second DSP chip is used to perform blind source separation on the dereverberated sound signal to obtain the speech information.

[0059] More specifically, in the dereverberation device, the dereverberation process involves performing a short-time Fourier transform (STFT) on the processed sound signal to convert the time-domain signal into a frequency-domain signal; then, a normalized least mean square (NLMS) adaptive algorithm is applied to perform multi-channel dereverberation, effectively eliminating environmental echoes and reverberation effects to obtain a dereverberated sound signal.

[0060] More specifically, in the second DSP chip, the blind source separation process is as follows: based on the OverIVA (Overdetermined Independent Vector Analysis) blind source separation algorithm, the dereverberated sound signal after dereverberation processing is decomposed, and by maximizing the independence between signal sources, the target speech component is separated and the background noise is suppressed to obtain the purified target speech signal, that is, the speech information.

[0061] In this embodiment, the speech information is enhanced based on the least squares GAN model in the information enhancement unit to obtain the clear speech information.

[0062] More specifically, the enhancement process is as follows: the generator (G) performs denoising and spectral compensation on the input speech information, and the discriminator (D) judges the generated result based on the least squares loss function. Through adversarial training between the generator and the discriminator, the generator (G) finally outputs clear speech information.

[0063] In this embodiment, the personal terminal includes an ambient sound detection unit, a hearing test unit, a control unit, a conversion unit, and an interaction unit. The ambient sound detection unit receives ambient sounds and determines whether the current environment meets the testing requirements. The hearing test unit performs a hearing test on the wearer and generates a hearing loss level report, which is a report on the user's hearing threshold. The control unit generates a compensation scheme based on the hearing loss level report and performs sound compensation on the bone conduction hearing aid based on the compensation scheme and the sound emission direction. The control unit also controls the sound pickup direction of the pickup module. The conversion unit receives the clear speech information and converts it into text information. The interaction unit provides human-computer interaction and displays the text information.

[0064] More specifically, in the control unit, the compensation scheme is generated as follows: based on the frequency adaptive gain model generated by the hearing loss level report, the amplification curve of the bone conduction hearing aid is dynamically adjusted to obtain the compensation scheme.

[0065] The specific process is as follows:

[0066] 1. Extracting core parameters of hearing loss

[0067] Two key data points can be obtained from the hearing loss level report:

[0068] Hearing thresholds at various test frequencies The test frequency covers 125Hz-8kHz (7 core frequency points are set according to octave: 125Hz, 250Hz, 500Hz, 1kHz, 2kHz, 4kHz, 8kHz). Representing users at frequency The minimum perceptible sound intensity (unit: dB HL).

[0069] Average hearing threshold By calculating the above 7 frequency points The arithmetic mean is obtained, reflecting the overall degree of hearing loss of the user.

[0070] 2. Construct a frequency-adaptive gain model and calculate the target gain.

[0071] Based on the NAL-NL2 prescription formula, a frequency adaptive gain model is constructed, and the target gain is calculated band by band. :

[0072]

[0073] Calculation logic: for each core frequency Substitute and The numerical value is used to obtain the target gain (unit: dB) that needs to be compensated for in this frequency band, ensuring that the gain is positively correlated with the degree of hearing loss of the user, and avoiding excessive amplification of normal hearing frequency bands.

[0074] 3. Spatial filtering optimization based on sound source direction

[0075] The angle of the sound source output by the control unit receiving direction judgment unit Spatial adaptation optimization of the target gain:

[0076] Design weight vector based on MVDR (Minimum Variance Distortionless Response) beamformer: ,in The noise covariance matrix is ​​calculated from the ambient noise collected in real time by the sound pickup module. The guide vector (relative to the angle of the sound source) match), for The conjugate transpose of;

[0077] Optimization objective: Based on the target gain, enhance the sound signal in the direction of the sound source and suppress background noise in the direction of the non-sound source, so that the amplification curve can simultaneously meet the dual requirements of "hearing compensation" and "directional enhancement".

[0078] 4. Dynamically adjust the parameters of the magnification curve.

[0079] The calculated target gain and spatial filtering weight for each frequency band This is converted into the amplification curve adjustment parameters for bone conduction hearing aids:

[0080] Frequency band matching: The 125Hz-8kHz frequency range is matched one-to-one with the hearing aid's built-in amplification channels (each channel covers 1-2 core frequency points), and a corresponding frequency band is assigned to each channel. value;

[0081] Safety threshold limit: Set the maximum gain threshold (maximum gain of a single frequency band shall not exceed 60dB) to avoid excessive gain from damaging the user's residual hearing;

[0082] Real-time dynamic updates: If the user's movement changes the direction of the sound source If the ambient noise level changes, or if the ambient noise detection unit detects a change in ambient noise intensity (e.g., noise ≥ 35 dB (A-weighted)), the model will recalculate. and weight vector Simultaneously adjust the magnification curve.

[0083] 5. Output the final compensation plan

[0084] Adjusted amplification curve parameters (including target gain and spatial filter weights for each frequency band) This is integrated into a complete compensation solution and transmitted to the audio output module of the bone conduction hearing aid to achieve "frequency band compensation for the user's hearing loss + directional enhancement for the sound source", ensuring that the user can clearly perceive the target speech in different environments.

[0085] More specifically, the hearing test process in the hearing test unit is as follows:

[0086] Several test audios of different frequencies and decibels are generated and sent to the user through the bone conduction hearing aid;

[0087] The user selects whether they can hear the test audio based on the prompts from the interactive unit;

[0088] Based on the user's selection, the hearing threshold at each frequency is calculated using cubic spline interpolation to generate the hearing loss level report.

[0089] It should be noted that the ambient noise level is confirmed to be below 35dB (A-weighted) by the ambient sound detection unit to ensure that the test conditions meet the standards; multiple frequency ranges from 125Hz to 8kHz are generated, as well as multiple decibel levels starting from 0dB HL and increasing in 5dB increments; based on the user's selection, the hearing threshold at each frequency is calculated using interpolation to generate the hearing loss level report.

[0090] Example 2

[0091] In this embodiment, the sound pickup module acquires ambient sound signals through a multi-microphone array. To improve the clarity and directional recognition capability of the speech signal, this system employs a combination of multi-level filtering and multi-channel blind source separation in the signal preprocessing stage.

[0092] First, the noise reduction unit uses low-pass and high-pass filters to perform frequency division processing on the surrounding ambient sound, where:

[0093] The first low-pass filter is used to suppress high-frequency noise; the high-pass filter is used to eliminate low-frequency noise; the two signals are phase-synchronized and superimposed by the first DSP chip to retain the main audio band; then the second low-pass filter filters out residual background noise to obtain the optimized intermediate frequency speech signal.

[0094] In the speech extraction stage, dereverberation processing and the OverIVA blind source separation algorithm are introduced. The specific process is as follows:

[0095] 1. Perform a short-time Fourier transform (STFT) on the filtered signal to obtain its frequency domain representation;

[0096] 2. Perform multi-channel dereverberation based on the Normalized Least Mean Square (NLMS) algorithm to eliminate spatial echo;

[0097] 3. The OverIVA blind source separation algorithm is adopted to decompose and separate the target speech components from the dereverberation signal by maximizing the independence between signal sources.

[0098] This process can effectively maintain the integrity of speech features under non-ideal noise and reverberation conditions, providing high-quality input for subsequent speech enhancement.

[0099] The speech enhancement module in this embodiment uses an improved least-squares generative adversarial network (LS-GAN) for speech information enhancement. Its core idea is to introduce a least-squares error term into the adversarial training of the generator and discriminator to mitigate gradient vanishing and improve generation quality.

[0100] The model structure includes:

[0101] Generator (G): Inputs speech features after blind source separation, outputs enhanced speech after denoising and spectral compensation;

[0102] Discriminator (D): Receives real speech and generated speech, and determines their authenticity based on the least squares loss function;

[0103] During training, the loss function is defined as:

[0104]

[0105] in, This refers to samples taken from a real, clean speech dataset, i.e., the desired clear speech signal; The term refers to samples sampled from noisy or unenhanced speech datasets, and in this invention specifically refers to speech information obtained after blind source separation. This indicates the enhanced speech generated by the generator after taking noisy speech z as input; and This represents the discriminator's judgment result on the input signal (real speech x or generated speech) G(z). The closer the output value is to 1, the higher the probability that the discriminator considers the input signal to be real and clean speech. It represents the mathematical expectation value, which is the average of the loss values ​​over all samples; and These represent the loss functions of the discriminator and the generator, respectively. Used to guide the discriminator in distinguishing between real and generated speech; Used to guide the generator to produce enhanced speech that is sufficient to "fool" the discriminator.

[0106] Through iterative optimization, the generator gradually learns the spectral distribution characteristics of real speech, thereby outputting high-fidelity, clear speech information. The enhanced speech signal is transmitted to a personal terminal for subsequent text conversion and interactive display.

[0107] The personal terminal integrates a hearing test module, using a bone conduction sound unit to play test sounds of different frequencies and intensities. The system automatically generates a hearing threshold report based on user feedback (able to hear / unable to hear) on the interactive interface, and performs personalized compensation calculations based on the results. This testing function can be implemented using appropriate algorithms.

[0108] The algorithm process includes:

[0109] 1. Dynamically generate test tone sequences with multiple frequencies and decibel levels. Specifically, based on ISO389 standard calibration, generate 7 frequency points in octave bands within the range of 125Hz-8kHz. Each frequency starts from 0dB HL and increments in 5dB steps, with a maximum output not exceeding 100dB HL.

[0110] 2. Based on the user's response, construct a hearing response matrix. Specifically, record the user's response (1 / 0) for each frequency-intensity combination, and use a double-ascent method to confirm the threshold.

[0111] 3. Calculate the hearing loss curves for each frequency band using least squares fitting. Specifically, use cubic spline interpolation to smooth the original threshold points and generate continuous hearing curves.

[0112] 4. The frequency adaptive gain model is generated by the control unit to dynamically adjust the amplification curve of the bone conduction hearing aid;

[0113] 5. By combining the sound source angle information output by the direction judgment unit, spatial filtering optimization is performed to achieve intelligent compensation of "sound source enhancement".

[0114] More specifically, the frequency adaptive gain model calculates the target gain for each frequency band based on the NAL-NL2 prescription formula. , ,in For frequency Hearing threshold, The average hearing threshold;

[0115] More specifically, the spatial filtering optimization process involves: combining the direction judgment unit's output sound source angle... Design the weight vector of the MVDR beamformer ,in The noise covariance matrix is... As the guide vector, for The conjugate transpose of .

[0116] The conversion unit employs an end-to-end deep speech recognition model (E2E ASR) to achieve high-precision text recognition based on enhanced speech signals. The recognition results are displayed in real-time on the interactive unit, allowing users to interact with the terminal via voice or text to perform hearing tests, environmental monitoring, and personalized settings.

[0117] The pickup module is installed on the bone conduction hearing aid to collect ambient sounds and extract clear speech information from them. The pickup module includes: a pickup device, a noise reduction unit, a voice extraction unit, an information enhancement unit, and a direction determination unit.

[0118] The sound pickup device is used to collect ambient sounds. In this embodiment, the sound pickup device can be a microphone array composed of several small microphones.

[0119] The noise reduction unit is used to process ambient noise to obtain a processed sound signal. In this embodiment, the noise reduction unit includes a first low-pass filter, a high-pass filter, a first DSP chip, and a second low-pass filter working together. The first low-pass filter has a cutoff frequency of 8kHz to filter out high-frequency noise (such as wind noise or electronic interference) in the ambient sound and extracts a high-frequency phase signal, which is then input to the first DSP chip. The high-pass filter has a cutoff frequency of 100Hz to filter out low-frequency noise (such as ambient vibration noise) and extracts a low-frequency phase signal, which is then input to the first DSP chip. The first DSP chip uses an adaptive digital signal processing algorithm to coherently superimpose the high-frequency and low-frequency phase signals to enhance the signal-to-noise ratio of the speech band (300Hz-3400Hz). The second low-pass filter further filters out residual noise (such as white noise) in the superimposed signal, outputting a clean processed sound signal. The filter can be the LFCV-2002+ model and supports dynamic parameter adjustment to adapt to different environments.

[0120] The voice extraction unit is used to extract speech information from the processed sound signal. The voice extraction unit includes a receiving device, a dereverberation device, and a second DSP chip. The receiving device receives the processed sound signal. The dereverberation device performs dereverberation processing on the processed sound signal to obtain a dereverberated sound signal. In this embodiment, the dereverberation processing is to eliminate the influence of echo. The method includes: first, performing a short-time Fourier transform (STFT) on the signal to convert the time-domain signal into a frequency-domain signal; then applying a normalized least mean square (NLMS) adaptive algorithm for multi-channel dereverberation to effectively eliminate environmental echo and reverberation effects, obtaining a dereverberated sound signal; the second DSP chip decomposes the dereverberated signal based on the OverIVA (Overdetermined Independent Vector Analysis) blind source separation algorithm: by maximizing the independence between signal sources, the target speech component is separated, and background noise is suppressed. This method is particularly suitable for multi-channel input of microphone arrays and can improve separation accuracy by combining sound source localization information.

[0121] The information enhancement unit is used to enhance the speech information to obtain clear speech information, and then transmits the clear speech information to the personal terminal. The information enhancement method includes enhancing the speech information based on a least squares GAN model to obtain clear speech information. In this embodiment, the information enhancement unit adopts a least squares generative adversarial network (LS-GAN) model, and its training process is optimized as follows:

[0122] Data preparation: Collect a large number of real speech datasets (such as TIMIT and LibriSpeech) and perform preprocessing (including pre-emphasis, frame segmentation and normalization).

[0123] Model Training: The generator receives the speech information after blind source separation and generates an enhanced signal through a convolutional neural network (CNN) structure; the discriminator judges the difference between the generated signal and the real speech based on the least squares loss function. During training, an alternating optimization strategy is used: first, the generator is fixed and the discriminator weights are updated; then, the discriminator is fixed and the generator weights are updated. The number of iterations is usually greater than 1000 rounds until the loss function converges (e.g., the discriminator loss is less than 0.1).

[0124] Enhanced Applications: After training, the model can process input speech in real time, output clear speech information with high signal-to-noise ratio through the generator, and dynamically adjust parameters to adapt to the speech characteristics of different users.

[0125] The direction determination unit is used to determine the direction of sound emission based on the adaptive beamforming method. In this embodiment, the direction determination unit collects multiple sound signals from the microphone array, processes the multiple sound signals using a multi-channel model, calculates the time delay difference and relative intensity difference of the signals, then uses an adaptive algorithm to dynamically weight the received multiple sound signals to improve the signal-to-noise ratio of the signal wavefront, and finally merges the weighted signals to estimate the direction of sound emission.

[0126] The personal terminal is used to control the bone conduction hearing aid for assisted hearing testing and also to control the sound pickup module. The personal terminal includes: an ambient sound detection unit, a hearing testing unit, a control unit, a conversion unit, and an interaction unit.

[0127] The ambient sound detection unit receives ambient sounds and determines whether the current environment is suitable for testing. In this embodiment, an ambient sound threshold is first set, and then compared with the ambient sounds collected by the sound pickup device. When the collected ambient sound is greater than the threshold, the interactive unit will prompt that the current environment is not suitable for testing. When the collected ambient sound is less than the threshold, the interactive unit will prompt the user to perform the test operation.

[0128] The hearing test unit is used to perform hearing tests on the wearer and generate a hearing loss level report. The hearing test method includes: generating several test audios of different decibels and sending the test audios to the user through a bone conduction hearing aid; the user selects whether they can hear the test audios according to the prompts of the interactive unit; and generating a hearing loss level report based on the user's selection, wherein the hearing loss level report is a report on the user's hearing threshold.

[0129] The control unit generates a compensation plan based on the hearing loss level report and performs sound compensation on the bone conduction hearing aid based on the compensation plan and the sound emission direction; the control unit also controls the sound pickup direction of the pickup module. The conversion unit receives clear speech information and converts it into text information. In this embodiment, a speech recognition algorithm is used to convert clear speech information into text information, and the interaction unit displays the text information to the user.

[0130] Example 3

[0131] In this embodiment, the hearing test steps for a personal terminal will be described in detail:

[0132] (1) Choose a suitable hearing test tool: Select a hearing test tool that is suitable for people with hearing impairments. This can be a specially designed hearing test application, device, or hardware. If necessary, ensure that the test tool is compatible with hearing aids.

[0133] (2) Conduct a hearing test: Perform a hearing test according to the instructions of the testing tool. This may include testing for sounds at different frequencies, word recognition, sentence comprehension, etc. Record the test results, including hearing level, type and degree of any hearing loss.

[0134] The hearing test unit is implemented through a software module. This module first initializes the audio parameters, and then cyclically generates pure tone signals at specified frequencies and decibels. After the signals are output through the bone conduction hearing aid, the module listens to the user's interactive input (such as touchscreen clicks) and records the minimum volume threshold that the user can perceive. Finally, it analyzes the data to generate a hearing report.

[0135] Hearing tests require control and precision to measure the hearing threshold of the test subject. Furthermore, factors such as audio correction and headphone calibration need to be considered to ensure the accuracy of the test.

[0136] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A hearing aid system based on artificial intelligence, characterized in that, The device includes a bone conduction hearing aid, a pickup module, and a personal terminal. The pickup module is installed on the bone conduction hearing aid and is used to collect ambient sounds and extract clear speech information from the ambient sounds. The personal terminal is used to control the bone conduction hearing aid to perform assisted hearing tests and to control the pickup module. The sound pickup module includes a sound pickup device, a noise reduction unit, a voice extraction unit, an information enhancement unit, and a direction determination unit. The sound pickup device is used to collect ambient sound. The noise reduction unit is used to process the ambient sound to obtain a processed sound signal. The voice extraction unit is used to extract speech information from the processed sound signal. The information enhancement unit is used to enhance the speech information to obtain clear speech information with a signal-to-noise ratio improvement of ≥15dB and a speech intelligibility score of ≥3.5 on a 5-point scale, and transmits the clear speech information to the personal terminal. The direction determination unit is used to determine the direction of sound emission based on an adaptive beamforming method. The personal terminal includes an ambient sound detection unit, a hearing test unit, a control unit, a conversion unit, and an interaction unit. The ambient sound detection unit receives ambient sounds and determines whether the current environment meets the testing requirements. The hearing test unit performs a hearing test on the wearer and generates a hearing loss level report, which is a report of the user's hearing threshold. The control unit generates a compensation scheme based on the hearing loss level report and performs sound compensation on the bone conduction hearing aid based on the compensation scheme and the sound emission direction. The control unit also controls the sound pickup direction of the pickup module. The conversion unit receives the clear speech information and converts it into text information. The interaction unit provides human-computer interaction and displays the text information.

2. The artificial intelligence-based hearing aid system according to claim 1, characterized in that, In the direction determination unit, multiple sound signals from the microphone array of the pickup device are collected, the time delay difference and relative intensity difference of the signals are calculated, and then the multiple received sound signals are dynamically weighted using the variable step size normalized least mean square algorithm to improve the signal-to-noise ratio of the wavefront. Finally, the weighted signals are merged to estimate the direction of sound emission.

3. The artificial intelligence-based hearing aid system according to claim 2, characterized in that, The voice extraction unit includes a receiving device, a dereverberation device, and a second DSP chip; the receiving device is used to receive the processed sound signal; the dereverberation device is used to perform dereverberation processing on the processed sound signal to obtain a dereverberated sound signal; the second DSP chip is used to perform blind source separation on the dereverberated sound signal to obtain the speech information.

4. The artificial intelligence-based hearing aid system according to claim 3, characterized in that, In the dereverberation device, the dereverberation process involves performing a short-time Fourier transform on the processed sound signal to convert the time-domain signal into a frequency-domain signal; then, a normalized least mean square adaptive algorithm is applied to perform multi-channel dereverberation, effectively eliminating environmental echoes and reverberation effects to obtain a dereverberated sound signal.

5. The artificial intelligence-based hearing aid system according to claim 3, characterized in that, In the second DSP chip, the specific process of blind source separation is as follows: based on the OverIVA blind source separation algorithm, the dereverberated sound signal after dereverberation processing is decomposed, and the target speech component is separated and the background noise is suppressed by maximizing the independence between signal sources, so as to obtain the purified target speech signal, that is, the speech information.

6. The artificial intelligence-based hearing aid system according to claim 3, characterized in that, The noise reduction unit includes a first low-pass filter, a high-pass filter, a first DSP chip, and a second low-pass filter. The first low-pass filter is used to filter out high-frequency noise in the ambient sound and input the high-frequency phase signal into the first DSP chip. The high-pass filter is used to filter out low-frequency noise in the ambient sound and input the low-frequency phase signal into the first DSP chip. The first DSP chip is used to superimpose the high-frequency phase signal and the low-frequency phase signal to obtain a superimposed signal. The second low-pass filter is used to filter out high-frequency residual noise in the superimposed signal to obtain the processed sound signal.

7. The artificial intelligence-based hearing aid system according to claim 6, characterized in that, In the information enhancement unit, the speech information is enhanced based on the least squares GAN model. The specific enhancement process is as follows: the generator G is used to denoise and compensate the spectrum of the input speech information, and the discriminator D judges the generated result based on the least squares loss function. Through adversarial training between the generator and the discriminator, the generator G finally outputs clear speech information.

8. The artificial intelligence-based hearing aid system according to claim 7, characterized in that, In the control unit, based on the frequency adaptive gain model generated from the hearing loss level report, the amplification curve of the bone conduction hearing aid is dynamically adjusted to obtain a compensation scheme. The specific process is as follows: S21: Extract hearing thresholds for each core frequency within the 125Hz-8kHz frequency band from the hearing loss level report. and average hearing threshold ; S22: Calculate the target gain for each frequency band based on the NAL-NL2 prescription formula. Construct a frequency-adaptive gain model; S23: Determine the angle of the sound source output by the unit based on the direction. Spatial filtering optimization is performed using the weight vector of the MVDR beamformer; S24: Convert the target gain and weight vector of each frequency band into the amplification curve adjustment parameters of the bone conduction hearing aid, adapt them to the built-in amplification channel of the hearing aid, set the maximum gain safety threshold, and update the amplification curve. S25: Integrate the adjusted amplification curve parameters into a complete compensation scheme and transmit it to the audio output module of the bone conduction hearing aid.

9. A hearing aid system based on artificial intelligence according to claim 8, characterized in that, The frequency adaptive gain model calculates the target gain for each frequency band based on the NAL-NL2 prescription formula. , ,in For frequency Hearing threshold, This represents the average hearing threshold.

10. A hearing aid system based on artificial intelligence according to claim 8, characterized in that, In the aforementioned hearing test unit, the hearing test process is as follows: Multiple test audios with different frequencies and decibels are generated and sent to the user through the bone conduction hearing aid; The user selects whether they can hear the test audio based on the prompts from the interactive unit; Based on the user's selection, the hearing threshold at each frequency is calculated using cubic spline interpolation to generate the hearing loss level report.

Images