Hearing aid equipment personalized noise reduction method and system based on neural network

Abstract

The embodiment of the invention discloses a personalized noise reduction method and system for hearing aid equipment based on a neural network, and relates to the technical field of artificial intelligence. The method comprises the following steps: acquiring a noisy voice signal acquired by hearing aid equipment; de-noising the noisy voice signal by using a lightweight CRNN hybrid model to obtain a noise-free voice signal, and providing the noise-free voice signal to a user; in response to the noise reduction effect evaluation of the user, adjusting the weight in the lightweight CRNN hybrid model to improve the noise reduction effect; and evaluating the corresponding voice signals before and after denoising according to the denoising effect, and continuously optimizing the lightweight CRNN hybrid model. According to the embodiment of the invention, adaptive real-time noise reduction can be realized.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of artificial intelligence technology, and in particular, to a method and system for personalized noise reduction of hearing aids based on neural networks. Background Art

[0002] At present, hearing aids and cochlear implants and other hearing aids play an important role in helping hearing-impaired people obtain sound information. However, there are many deficiencies in existing noise reduction technologies:

[0003] 1. Traditional noise reduction methods mostly rely on manually designed features, which are difficult to adapt to complex and changeable noise environments, and have poor processing effects on complex noises such as non-stationary noises and multi-person speech interferences, resulting in low speech clarity and making it difficult for hearing-impaired people to accurately understand the content of the speech.

[0004] 2. Lack of adaptive optimization ability, unable to make real-time adjustments according to the hearing characteristics of different users and the changes in the usage environment, unable to meet personalized needs, and there are large differences in the experiences of different users when using the same device.

[0005] 3. The computational complexity of the noise reduction algorithm is high, making it difficult to achieve real-time noise reduction, affecting the smoothness of device use, and bringing inconvenience to the real-time communication of hearing-impaired people. Summary of the Invention

[0006] Embodiments of the present invention provide a method and system for personalized noise reduction of hearing aids based on neural networks to solve at least one of the above problems.

[0007] In a first aspect, embodiments of the present invention provide a method for personalized noise reduction of hearing aids based on neural networks, including:

[0008] Obtain a noisy speech signal collected by a hearing aid;

[0009] Use a lightweight CRNN hybrid model to denoise the noisy speech signal to obtain a noise-free speech signal and provide it to the user;

[0010] In response to the user's evaluation of the noise reduction effect, adjust the weights in the lightweight CRNN hybrid model to improve the noise reduction effect;

[0011] Continuously optimize the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation.

[0012] In a second aspect, embodiments of the present invention provide a system for personalized noise reduction of hearing aids based on neural networks, including:

[0013] A hearing aid for obtaining a noisy speech signal collected by the hearing aid;

[0014] A control unit that uses a lightweight CRNN hybrid model to denoise the noisy speech signal, obtains a noise-free speech signal, and provides it to the user;

[0015] The APP side is used to collect the user's evaluation of the noise reduction effect;

[0016] The control unit is further configured to adjust the weights in the lightweight CRNN hybrid model in response to the user's evaluation of the noise reduction effect to improve the noise reduction effect; and continuously optimize the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, where the electronic device includes:

[0018] One or more processors;

[0019] A memory for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the personalized noise reduction method for hearing aid devices based on a neural network according to any embodiment.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the personalized noise reduction method for hearing aid devices based on a neural network according to any embodiment.

[0022] In summary, this embodiment provides a personalized noise reduction method and system for hearing aid devices based on a neural network. Combining a lightweight CRNN hybrid model with an APP feedback mechanism, it realizes personalized real-time noise reduction for hearing aids and cochlear implants. Specifically, this embodiment uses the convolutional layer of the CRNN hybrid model to automatically extract the features of noise and speech, the recurrent layer to process time series information, and end-to-end learning of the mapping from noise to clean speech, avoiding manual feature design; collects the patient's evaluation of the noise reduction effect through the APP, dynamically adjusts the processing weights of each frequency or various types of noise in the model, realizes adaptive personalized noise reduction, and continuously optimizes the noise reduction database.

[0023] This embodiment can achieve the following beneficial effects:

[0024] 1. The noise reduction effect is significantly improved: Based on end-to-end modeling and the non-linear processing ability of the CRNN hybrid model, it can effectively separate complex noise and speech, significantly improve the speech clarity in complex environments, help hearing-impaired people hear and understand sounds more clearly, and enhance the communication experience.

[0025] 2. High degree of personalization: By collecting user evaluations through the APP and dynamically adjusting the noise reduction parameters according to the feedback of different users, personalized needs are met, and users with different hearing characteristics and usage habits can obtain a more suitable noise reduction effect for themselves.

[0026] 3. Achieve extremely high real-time performance: The lightweight CRNN hybrid model has low computational complexity and can achieve real-time noise reduction, ensuring the smoothness of the device during use and meeting the needs of hearing-impaired people for real-time communication.

[0027] 4. Have the ability of continuous optimization: Use user evaluation data to continuously quantify and distill the noise reduction database, enabling the model to continuously learn and optimize, adapt to the changing noise environment and user needs, and extend the service life and applicability of the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0029] Figure 1 is a flowchart of a personalized noise reduction method for a hearing aid device based on a neural network provided by an embodiment of the present invention;

[0030] Figure 2 is a schematic structural diagram of a personalized noise reduction system for a hearing aid device based on a neural network provided by an embodiment of the present invention;

[0031] Figure 3 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0033] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0034] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", "connected to" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0035] Figure 1 is a flowchart of a method for personalized noise reduction of a hearing aid device based on a neural network provided by an embodiment of the present invention. This method is executed by an electronic device. As Figure 1 shown, this method specifically includes:

[0036] S110. Obtain the noisy speech signal collected by the hearing aid device.

[0037] The hearing aid device includes a hearing aid or a cochlear implant, and the microphone of the hearing aid device collects the speech signal containing noise in real time.

[0038] S120. Denoise the noisy speech signal by using a lightweight CRNN hybrid model to obtain a noise-free speech signal.

[0039] The noisy speech signal is input into the lightweight CRNN hybrid model in real time. The model processes the signal through end-to-end learning, uses the convolutional layer and the recurrent layer to process the signal, outputs the clean speech signal after removing the noise, and transmits it to the user after subsequent processing such as amplification to achieve real-time noise reduction.

[0040] Specifically, the lightweight CRNN hybrid model includes a convolutional neural network (CNN) layer and a recurrent neural network (RNN) layer. Among them, the CNN layer is used to extract the spatial features of the input audio signal, capable of quickly capturing the noise and speech features in different frequency bands; the RNN layer, such as the long short-term memory network (LSTM) or gated recurrent unit (GRU), is responsible for processing the time series information of the audio, memorizing the long-term and short-term dependencies, and accurately separating speech from noise. The model is designed to be lightweight, reducing the number of parameters and the computational complexity to meet the requirements of real-time noise reduction.

[0041] After the noisy speech signal is input into the lightweight CRNN hybrid model, the neural convolutional network is first used to extract the features of the noisy speech signal at each moment, obtaining a spatial feature time series; the recurrent neural network is used to process the spatial feature time series to obtain the noise-free speech signal at each moment; after amplifying the noise-free speech signal, it is transmitted to the user through the hearing aid device.

[0042] Optionally, the CRNN hybrid model can also be replaced by a model based on the Transformer architecture. The Transformer model has powerful parallel computing capabilities and self-attention mechanisms, performs better in processing long-sequence audio data and complex noise, and is expected to further improve the noise reduction effect and processing efficiency.

[0043] S130. In response to the user's evaluation of the noise reduction effect, adjust the weights in the lightweight CRNN hybrid model to improve the noise reduction effect.

[0044] Optionally, a dedicated APP is designed in this embodiment, which is connected to the hearing aid or cochlear implant through wireless communication methods such as Bluetooth. The APP provides a simple and intuitive interaction interface, and patients can evaluate the current noise reduction effect in real time, such as speech clarity, noise residue level, etc. The APP sends the patient's evaluation data to the control unit of the hearing aid device for adjusting the parameters of the noise reduction model.

[0045] During the user's use, the noise reduction effect is evaluated through the APP. The APP transmits the evaluation data to the control unit, and the control unit adjusts the processing weights of the model according to the evaluation results. For example, if the user feedbacks that the noise removal in the low-frequency band is not thorough, the control unit increases the weight of the low-frequency band noise processing; if the speech clarity in the high-frequency band is insufficient, the processing parameters in the high-frequency band are appropriately adjusted. By continuously collecting evaluation data and adjusting the weights, personalized noise reduction is achieved.

[0046] Optionally, the collection of user evaluations can also be achieved through physiological index monitoring. For example, by monitoring the changes in physiological indexes such as the user's heart rate and blood pressure through the sensors on the wearing device, combined with the user's subjective evaluation, it can more comprehensively reflect the impact of the noise reduction effect on the user and provide more accurate data basis for adaptive adjustment.

[0047] S140. Evaluate the speech signals before and after denoising corresponding to the denoising effect evaluation, and continuously optimize the lightweight CRNN hybrid model.

[0048] The control unit is also responsible for managing the denoising database, quantifying and distilling new denoising data, and updating the database so that the model can continuously learn new noise patterns and user preferences.

[0049] Optionally, the control unit quantifies the denoising data and the corresponding user evaluation after each adjustment, extracts key information, performs data distillation, and removes redundant information; then, stores the processed data in the denoising database to provide richer and more accurate data support for the subsequent training and optimization of the model, enabling the model to better adapt to different users and complex environments.

[0050] In a specific embodiment, the lightweight CRNN hybrid model can be continuously optimized by a reinforcement learning method. Specifically, taking the lightweight CRNN hybrid model as the Actor network and the denoising effect evaluation as the reward function, a reinforcement learning model is constructed; the Critic network in the model can adopt a fully connected layer, and the model parameters are continuously updated in each sample training.

[0051] Specifically, the speech signal before each denoising is used as the state variable. After being input into the Acotor network, the state features before the last layer of the network are extracted; the state features are input into the Critic network, and the Q values of each action variable (where each action variable is each denoising frequency band) are output; the reward value is calculated according to the denoising effect evaluation, positive rewards are given to the actions with good evaluation effects, negative rewards are given to the actions with poor evaluation effects, and the model parameters are updated according to each Q value and the reward function, so that the model parameters continuously improve in the direction of excellent denoising effect.

[0052] It should be noted that the user data involved in this application are all information and data authorized by the users or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0053] In summary, this embodiment provides a personalized noise reduction method for hearing aids based on neural networks. By combining a lightweight CRNN hybrid model with an APP feedback mechanism, personalized real-time noise reduction for hearing aids and cochlear implants is achieved. Specifically, in this embodiment, the convolutional layer of the CRNN hybrid model is used to automatically extract the features of noise and speech, the recurrent layer processes time series information, and end-to-end learning is performed on the mapping from noise to clean speech, avoiding manually designed features; the APP collects the user's evaluation of the noise reduction effect, dynamically adjusts the processing weights of each frequency or various types of noise in the model, realizes adaptive personalized noise reduction, and continuously optimizes the noise reduction database.

[0054] This embodiment can achieve the following beneficial effects:

[0055] 1. Significantly improved noise reduction effect: Based on end-to-end modeling and the non-linear processing ability of the CRNN hybrid model, complex noise and speech can be effectively separated, significantly improving speech clarity in complex environments, helping hearing-impaired people hear and understand sounds more clearly, and enhancing the communication experience.

[0056] 2. High degree of personalization: By collecting user evaluations through the APP and dynamically adjusting the noise reduction parameters according to the feedback of different users, personalized needs are met, and users with different hearing characteristics and usage habits can obtain a more suitable noise reduction effect for themselves.

[0057] 3. Extremely high real-time performance: The lightweight CRNN hybrid model has a low computational complexity and can achieve real-time noise reduction, ensuring the smoothness of the device during use and meeting the real-time communication needs of hearing-impaired people.

[0058] 4. Ability to continuously optimize: Using user evaluation data to continuously quantify and distill the noise reduction database enables the model to continuously learn and optimize, adapt to changing noise environments and user needs, and extend the service life and applicability of the device.

[0059] Figure 2 It is a schematic structural diagram of a personalized noise reduction system for hearing aids provided by an embodiment of the present invention. As Figure 2 shown, the system includes:

[0060] A hearing aid for acquiring a noisy speech signal collected by the hearing aid;

[0061] A control unit that uses a lightweight CRNN hybrid model to denoise the noisy speech signal to obtain a noise-free speech signal and provide it to the user;

[0062] The APP side for collecting the user's evaluation of the noise reduction effect;

[0063] The control unit is further configured to adjust the weights in the lightweight CRNN hybrid model in response to the user's evaluation of the noise reduction effect to improve the noise reduction effect, and continuously optimize the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation.

[0064] This system is based on the same inventive concept as any of the above method embodiments. Any limitation in the above methods is applicable to this embodiment and can achieve the same beneficial effects as the above methods.

[0065] Figure 3 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. As Figure 3 shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63. The number of processors 60 in the device can be one or more. Figure 3 Here, one processor 60 is taken as an example. The processor 60, the memory 61, the input device 62, and the output device 63 in the device can be connected through a bus or other means. Figure 3 Here, the connection through the bus is taken as an example.

[0066] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules corresponding to the personalized noise reduction method for hearing aid devices based on neural networks in the embodiments of the present invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, implements the above-mentioned personalized noise reduction method for hearing aid devices based on neural networks.

[0067] The memory 61 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function. The data storage area can store data created according to the use of the terminal, etc. In addition, the memory 61 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 61 can further include a memory remotely set relative to the processor 60, and these remote memories can be connected to the device through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0068] The input device 62 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the device. The output device 63 can include a display device such as a display screen.

[0069] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the personalized noise reduction method of the hearing aid device based on a neural network in any embodiment.

[0070] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program can be used by or combined with an instruction execution system, apparatus, or device.

[0071] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium can send, propagate, or transmit a program for use by or combined with an instruction execution system, apparatus, or device.

[0072] The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0073] Computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the C language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, execute as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0074] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A personalized noise reduction method for a hearing aid device based on a neural network, characterized in that, including: Obtaining a noisy speech signal collected by a hearing aid device; Denosing the noisy speech signal by using a lightweight CRNN hybrid model to obtain a noise-free speech signal and providing it to the user; Responding to the user's noise reduction effect evaluation, adjusting the weights in the lightweight CRNN hybrid model to improve the noise reduction effect; Continuously optimizing the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation.

2. The method according to claim 1, characterized in that, The lightweight CRNN hybrid model includes a convolutional neural network layer and a recurrent neural network layer; The using the lightweight CRNN hybrid model to denoise the noisy speech signal to obtain a noise-free speech signal and providing it to the user includes: Using the neural convolutional network to extract features from the noisy speech signal at each moment to obtain a spatial feature time series; Using the recurrent neural network to process the spatial feature time series to obtain a noise-free speech signal at each moment.

3. The method according to claim 1, characterized in that The responding to the user's noise reduction effect evaluation and adjusting the weights in the lightweight CRNN hybrid model includes: Responding to the user's noise reduction effect evaluation, extracting the speech frequency band to be optimized from the evaluation; Increasing the processing weight of the speech frequency band in the lightweight CRNN hybrid model.

4. The method according to claim 1, wherein Continuously optimizing the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation includes: Taking the lightweight CRNN hybrid model as the Actor network and the noise reduction effect evaluation as the reward function to construct a reinforcement learning model; Using the speech signals before and after denoising corresponding to each noise reduction effect evaluation as samples to train the reinforcement learning model, and using the reward function to improve the model parameters in the direction of excellent noise reduction effect during the training process.

5. The method according to claim 1, wherein The noise reduction effect evaluation includes speech clarity and noise residue degree.

6. The method according to claim 1, wherein The noise reduction effect evaluation comes from the APP terminal wirelessly connected to the hearing aid device.

7. The method according to claim 1, wherein The using the lightweight CRNN hybrid model to denoise the noisy speech signal to obtain a noise-free speech signal and providing it to the user further includes: After amplifying the noise-free speech signal, transmitting it to the user through the hearing aid device.

8. A personalized noise reduction system for a hearing aid device based on a neural network, characterized in that, including: A hearing aid device for obtaining a noisy speech signal collected by the hearing aid device; A control unit for denosing the noisy speech signal by using a lightweight CRNN hybrid model to obtain a noise-free speech signal and providing it to the user; An APP terminal for collecting the user's noise reduction effect evaluation; The control unit is further configured to respond to the user's noise reduction effect evaluation, adjust the weights in the lightweight CRNN hybrid model to improve the noise reduction effect; and continuously optimize the lightweight CRNN hybrid model according to the speech signals before and after denoising corresponding to the noise reduction effect evaluation.

9. An electronic device, characterized in that, including: One or more processors; A memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the neural network-based personalized noise reduction method for hearing aid devices according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the program is executed by a processor, it implements the method for personalized noise reduction of a hearing aid device based on a neural network according to any one of claims 1-8.

Images