Hearing aid device using artificial intelligence

Abstract

The present invention provides a hearing aid device comprising two earplug bodies, each of the two earplug bodies comprising: a microphone for converting a sound wave of an ambient sound into an electrical signal to generate an ambient sound signal; a speaker for converting an electrical signal of a sound into a sound wave; and a control unit for processing the ambient sound signal to generate an electrical signal of the sound, wherein the control unit comprises: an artificial intelligence sound analysis module for performing artificial intelligence analysis on the ambient sound signal to separate a voice signal and a non-voice signal other than the voice signal; and an output sound signal generation module for increasing the volume of the voice signal separated and outputted by the artificial intelligence sound analysis module, thereby generating an output sound signal in which the volume of the non-voice signal is lowered, and transmitting same to the speaker.

Description

Hearing assistance devices using artificial intelligence

[0001] The present invention relates to a hearing assistance device that can be effectively used in noisy environments such as dental clinics.

[0002] Creating a comfortable and relaxing environment is paramount when treating patients. While a quiet environment can be beneficial depending on the treatment method, in the dental field, the sharp, high-frequency noises generated during treatment can cause discomfort or even fear in patients. However, measures to protect patients from exposure to this high-frequency noise are inadequate.

[0003] Beyond the medical field, such as dentistry, the importance of hearing protection for workers is growing in noisy industrial settings like military bases and construction sites. To address this, workers currently primarily use shielding earplugs. However, these earplugs block not only noise but also alarms and voices from other workers within the facility, causing communication problems and potentially contributing to safety accidents. This problem is particularly acute for those with hearing impairments.

[0004] Meanwhile, active noise control (ACTIVE NOISE CONTROL) technology utilizes a microphone attached to headphones or earphones to detect ambient noise and generate destructive interference within the noise-canceling circuit to cancel out the noise. Previously, active noise control technology was incorporated into headphones or earphones for multimedia enjoyment, and significant development has not been made to apply it to other environments.

[0005] The purpose of the present invention is to provide a hearing assistance device that enables conversation while eliminating noise in noisy environments such as medical fields such as dentistry and surgery, and industrial fields such as military units and construction sites.

[0006] In order to achieve the above object of the present invention, according to one aspect of the present invention, there is provided a hearing assistance device including two earplug bodies, each of the two earplug bodies having a microphone for converting sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker for converting the electrical signals of sound into sound waves, and a control unit for processing the ambient sound signals to generate the electrical signals of sound, wherein the control unit has an artificial intelligence sound analysis module for performing artificial intelligence analysis on the ambient sound signals to distinguish and separate voice signals from non-voice signals that are not voice signals, and an output sound signal generation module for generating an output sound signal that increases the volume of the voice signal output by being separated from the artificial intelligence sound analysis module and decreases the volume of the non-voice signal, and transmitting the generated output sound signal to the speaker.

[0007] The present invention achieves all of the aforementioned objectives. Specifically, by using artificial intelligence sound analysis to separate speech and non-speech signals from ambient sound signals and processing them separately, the volume of speech signals is increased and the volume of non-speech signals is decreased, thereby enabling conversation even in noisy environments such as dental clinics.

[0008] FIG. 1 is a perspective view illustrating a hearing aid device according to one embodiment of the present invention.

[0009] Figure 2 is an exploded perspective view of the hearing aid device illustrated in Figure 1.

[0010] FIG. 3 is a block diagram showing the configuration of the earplug body in the hearing aid device illustrated in FIGS. 1 and 2.

[0011] Figure 4 is a block diagram showing the configuration of the control unit illustrated in Figure 3.

[0012] FIG. 5 is a diagram showing three operating modes of the hearing aid device illustrated in FIGS. 1 and 2.

[0013] Hereinafter, the configuration and operation of an embodiment of the present invention will be described in detail with reference to the drawings.

[0014] A hearing assistance device according to one embodiment of the present invention is illustrated in a perspective view in FIG. 1 and in an exploded perspective view in FIG. 2. Referring to FIGS. 1 and 2, a hearing assistance device (100) according to one embodiment of the present invention includes two earplugs (110) and a necklace means (190) for connecting the two earplugs (110) so that they can be worn around a user's neck. The hearing assistance device (100) operates to intensively remove ambient noise, facilitate conversation in noisy situations, or enable hearing of sound information provided by an external device.

[0015] Each of the two earplugs (110) is used by being inserted into each of the user's two ears. Each of the two earplugs (110) has an earplug body (120) and an earcap (180) that is detachably connected to the earplug body (120).

[0016] The earplug body (120) has an extension portion (121) that extends in a columnar shape. A necklace means (190) is connected to the extension portion (121). Although not shown, the earplug body (120) has a rechargeable secondary battery that supplies the power required for operation. Charging of the secondary battery provided in the earplug body (120) is performed with the necklace means (190) removed.

[0017] FIG. 3 illustrates a block diagram of the configuration of an earplug body (120). Referring to FIG. 3, the earplug (120) includes an operating unit (122) for selecting an operating mode of a hearing aid device (100 of FIGS. 1 and 2), a microphone (130) for converting sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker (140) for converting electrical signals of sound into sound waves so that a user can hear them with his or her ears, a communication unit (145) for communicating with an external device (A), and a control unit (150) for controlling the overall operation of the earplug body (120).

[0018] The operating unit (122) is for operating the hearing aid device (100 of FIGS. 1 and 2), and the user can select the operating mode of the hearing aid device (100 of FIGS. 1 and 2) by using the operating unit (122). In the present embodiment, the operating unit (122) is described as being formed in the form of a button on the outside of the ear plug body (120) as illustrated in FIGS. 1 and 2. The operating mode selection signal by the operating unit (122) is transmitted to the control unit (150).

[0019] The microphone (130) converts sound waves from ambient sounds into electrical signals to generate ambient sound signals. The microphone (130) comprises a typical microphone configuration. Ambient sounds can be divided into voices, such as human voices, and non-voices. Ambient sounds can also include noise. The ambient sound signals generated by the microphone (130) are transmitted to the control unit (150).

[0020] The speaker (140) converts the electrical signal of sound transmitted from the control unit (150) into sound waves so that the user can hear it. The speaker (140) includes a configuration of a typical speaker.

[0021] The communication unit (145) connects the hearing aid device (100 of FIGS. 1 and 2) to an external device (A), such as a smartphone, so that the device can communicate with it. The communication unit (145) exchanges control signals and data with the control unit (150). The communication unit (145) can receive sound data, such as a sound source, from the external device (A). In this embodiment, the communication unit (145) is described as utilizing a short-range wireless communication technology, such as Bluetooth.

[0022] The control unit (150) controls the overall operation of the earplug body (120). FIG. 4 illustrates the configuration of the control unit (150) as a block diagram. Referring to FIG. 4, the control unit (150) includes an operation mode confirmation unit (152) that confirms the operation mode of the hearing aid device (100 of FIGS. 1 and 2), a noise removal module (155) that removes noise from an ambient sound signal output from a microphone (130 of FIG. 3), an artificial intelligence sound analysis module (160) that performs artificial intelligence analysis on an ambient sound signal output from a microphone (130 of FIG. 3), and an output sound signal generation module (165) that generates an output sound signal by adjusting the volume of a voice signal and a noise signal that are separated and output from the artificial intelligence sound analysis module (160).

[0023] The operation mode confirmation unit (152) confirms the operation mode of the hearing aid device (100 of FIGS. 1 and 2). The operation mode confirmation unit (152) confirms the operation mode selection signal output by the operating unit (122 of FIG. 3) to confirm the operation mode of the hearing aid device (100 of FIGS. 1 and 2). FIG. 5 shows three operation modes of the hearing aid device (100 of FIGS. 1 and 2). Referring to FIG. 5, the hearing aid device (100 of FIGS. 1 and 2) operates in one of three operation modes: a focused noise cancellation mode (M1), a conversation support mode (M2), and an external device connection mode (M3). The focused noise cancellation mode (M1) and the conversation support mode (M2) can be switched and changed. External device connection mode (M3) uses a Bluetooth connection, and either the focused noise cancellation mode (M1) or the conversation focus mode (M2) can operate within the external device connection mode (M3).

[0024] In the focused noise cancellation mode (M1), the hearing aid device (100 in FIGS. 1 and 2) operates to remove noise from the surrounding sounds.

[0025] In conversation support mode (M2), the hearing aid device (100 in FIGS. 1 and 2) operates to enable smooth conversation in noisy situations.

[0026] In the external device connection mode (M3), the hearing aid device (100 in FIGS. 1 and 2) operates to output sound data transmitted from the external device (A).

[0027] Referring to FIG. 4, the noise removal module (155) removes noise from an ambient sound signal output from a microphone (130 of FIG. 3) to generate a noise removal sound signal. The noise removal sound signal generated by the noise removal module (155) is transmitted to a speaker (140 of FIG. 3) and converted into a sound wave and output. The noise removal module (155) may use a low pass filter (LPF) or a notch filter. The low pass filter (LPF) passes the low frequency band of the ambient sound signal and blocks the high frequency band, thereby reducing or blocking high frequency noise distributed in a relatively high frequency band. The notch filter is a filter that removes only components of a specific frequency band. The frequency band that the noise removal module (155) removes may be set differently depending on the environment in which the hearing aid device (100 of FIGS. 1 and 2) is used. When the hearing aid device (100 of FIGS. 1 and 2) is used in a dental clinic, it is preferable that the noise reduction filter module (155) be set to block high frequency bands of about 3500 Hz or higher. The noise reduction module (155) can function when the hearing aid device (100 of FIGS. 1 and 2) is operated in a focused noise reduction mode (M1 of FIG. 5).

[0028] The artificial intelligence sound analysis module (160) performs artificial intelligence analysis on the ambient sound signal output from the microphone (130 of FIG. 3) to distinguish and separate voice signals from non-voice signals that are not voice signals. In the artificial intelligence sound analysis module (160), non-voice signals excluding voice signals are recognized as noise signals. The voice signals and noise signals separated by the artificial intelligence sound analysis module (160) are transmitted to the output sound signal generation module (165). The artificial intelligence sound analysis module (160) is equipped with an artificial intelligence sound analysis model (162) that performs artificial intelligence analysis on the ambient sound signal to separate voice signals from noise signals. The artificial intelligence sound analysis module (160) can function when the hearing assistance device (100 of FIGS. 1 and 2) operates in the focused noise removal mode (M1 of FIG. 5) and the conversation support mode (M2 of FIG. 5).

[0029] The artificial intelligence sound analysis model (162) is created by deep learning on collected sound data. The learning process is explained as follows using a dental case as an example.

[0030] First, as a first step, dental hospitals collect voice data by treatment type / sound generation type (suction, handpiece sound, baby crying sound, etc.) / department (conservative dentistry, pediatric dentistry, etc. different treatment environments) and conduct preliminary classification and tagging (Step 1).

[0031] Next, in the second step, only the medical staff's voices are classified from the actual field data, which is a mixture of collected voice and sound data. At this time, the medical staff's voices can be tagged by distinguishing the start and end points on a timeline or by frequency. For medical staff voice classification, after distinguishing whether the voices are from a medical staff or not, the medical staff's voices are converted to text (sound to text), selected and organized by sentence, and then tagged. For noise classification, after distinguishing whether it is noise or not, the type of dental noise is determined and tagged.

[0032] Next, in Step 3, a deep learning algorithm is developed based on the tagged data. Specifically, an environmental noise classification algorithm based on a convolutional neural network (CNN) is developed to improve speech discrimination and intelligibility in noisy environments. The input waveform is imaged in Hz over time, and this is used to classify speech and noise waveforms. This is used for the following two functions.

[0033] First, it is used to develop an algorithm that can classify the voices of medical staff and various noises of dental treatment.

[0034] Next, it can be utilized as a key solution for voice activity detection.

[0035] Residual convolutional neural networks (RCNs) are recommended. Residual learning approaches effectively address the problems of increased computational complexity and vanishing / explosive gradients by adding residual connections to specific layers, demonstrating superior performance even in deep neural network structures. Therefore, a method combining a CNN with residual learning techniques, such as a residual CNN, is used to identify noise.

[0036] The output sound signal generation module (165) is separated from the artificial intelligence sound analysis module (160) and generates an output sound signal by adjusting the volume of the output voice signal and the volume of the noise signal, respectively. The output sound signal generation module (165) increases the volume of the voice signal and decreases the volume of the noise signal and outputs it. The output sound signal generated by the output sound signal generation module (165) is transmitted to a speaker (140 of FIG. 3) and converted into a sound wave and output. The artificial intelligence sound analysis module (160) can function when the hearing assistance device (100 of FIGS. 1 and 2) operates in a conversation support mode (M2 of FIG. 5). Accordingly, even in a noisy surrounding environment, the noise is heard small and the voice is heard large, so that a conversation can be conducted smoothly.

[0037] Referring to FIGS. 1 and 2, an ear cap (180) is detachably connected to the earplug body (120). The ear cap (180) may be replaced with a variety of sizes to suit the user's characteristics. The ear cap (180) may be configured in a configuration that can be commonly used in earphones.

[0038] A necklace means (190) connects two earplugs (110) so that they can be hung around the user's neck. The necklace means (190) has two connecting parts (192) that are connected to the earplug body (120) and a connecting string (196) that connects the two connecting parts (192).

[0039] Each of the two connecting portions (192) is detachably connected to each of the two earplug bodies (120). The connecting portion (192) is generally in the shape of a ring and is connected to the earplug body (120) by wrapping around the extension portion (121) of the earplug body (120). The connecting portion (192) is made of an elastic material.

[0040] A connecting line (196) connects two connecting parts (192). The connecting line (196) is hung around the user's neck.

[0041] Now, referring to FIGS. 1 to 5, the operation of the hearing aid device (100) according to its operating mode will be described. The hearing aid device (100) operates in one of three operating modes: a focused noise cancellation mode (M1), a conversation support mode (M2), and an external device connection mode (M3).

[0042] First, the focused noise reduction mode (M1) will be described. The focused noise reduction mode (M1) is performed by the operation mode confirmation unit (152) confirming the focused noise reduction mode selection signal output by the operation unit (122) operated by the user. In the focused noise reduction mode (M1), the noise reduction module (155) operates to block a set frequency band, thereby performing the function. In addition, in the focused noise reduction mode (M1), the artificial intelligence sound analysis module (160) and the output sound signal generation module (165) also operate together. The focused noise reduction mode (M1) is useful when the ambient noise is very severe and noise needs to be blocked intensively.

[0043] Next, the conversation support mode (M2) will be described. The conversation support mode (M2) is performed by the operation mode confirmation unit (152) confirming the conversation support mode selection signal output by the operation unit (122) operated by the user. In the conversation support mode (M2), the artificial intelligence sound analysis module (160) operates to separate the voice signal and the noise signal from the ambient sound signal, and the output sound signal generation module (165) operates to increase the volume of the voice signal and decrease the volume of the noise signal, thereby performing the operation. The conversation support mode (M2) is useful when smooth conversation is required in a noisy surrounding environment.

[0044] Next, the external device connection mode (M3) will be described. The external device connection mode (M3) is performed by the operation mode confirmation unit (152) confirming the external device connection mode selection signal output by the operation unit (122) operated by the user. The external device connection mode (M3) is performed by the communication unit (145) connecting the external device (A) to enable communication. In the external device connection mode (M3), sound data such as a sound source is received from the external device (A) and output through the speaker (140). The external device connection mode (M3) is useful when the user wants to listen to music to concentrate on work or to achieve psychological stability during dental treatment. In addition, the focused noise reduction mode (M1) and the conversation support mode (M2) can be used together in the external device connection mode (M3). For the psychological stability of the user, it is preferable that the focused noise reduction mode (M1) be used together with the external device connection mode (M3). For example, in a dental setting, listening to music in a quiet environment can reduce dental fear, and in an industrial / work environment, it can reduce perceived noise.

[0045] While the present invention has been described through the above examples, the present invention is not limited thereto. The above examples may be modified or altered without departing from the spirit and scope of the present invention, and those skilled in the art will recognize that such modifications and variations also fall within the scope of the present invention.

Claims

1. Includes two earplug bodies, Each of the above two earplug bodies has a microphone that converts sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker that converts electrical signals of sound into sound waves, and a control unit that processes the ambient sound signals to generate electrical signals of sound. The above control unit is equipped with an artificial intelligence sound analysis module that performs artificial intelligence analysis on the above ambient sound signal to distinguish and separate voice signals from non-voice signals that are not voice signals, and an output sound signal generation module that generates an output sound signal that increases the volume of the voice signal separated from the artificial intelligence sound analysis module and lowers the volume of the non-voice signal and transmits the generated output sound signal to the speaker. Hearing assistance devices.

2. In claim 1, The above artificial intelligence sound analysis module is provided with an artificial intelligence sound analysis model generated by machine learning to separate speech signals and non-speech signals from the surrounding sound signals. Hearing assistance devices.

3. In claim 2, The above machine learning is performed based on a residual convolutional neural network. Hearing assistance devices.

4. In claim 1, The above control unit further comprises a noise removal module that removes noise from the ambient sound signal to generate a noise removal signal and transmits the noise removal signal to the speaker. Hearing assistance devices.

5. In claim 4, The above noise removal module comprises a low-pass filter or a notch filter. Hearing assistance devices.

6. In claim 1, The above earplug body further comprises a communication unit for communication with an external device. The above control unit transmits the sound source transmitted from the external device to the speaker. Hearing assistance devices.

7. In claim 1, The above earplug body further comprises an operating unit that allows the user to select an operating mode and a communication unit for communication with an external device. The above control unit further comprises an operation mode confirmation unit that confirms an operation mode selection signal output by the operation unit, and a noise removal module that removes noise from the ambient sound signal to generate a noise removal signal and transmits the noise removal signal to the speaker. Depending on the above operation mode selection signal, it operates in one of the selected modes: focused noise cancellation mode, conversation support mode, and external device connection mode. In the above-mentioned focused noise removal mode, the noise removal module, the artificial intelligence sound analysis module and the output sound signal generation module operate. In the above conversation support mode, the artificial intelligence sound analysis module and the output sound signal generation module operate. In the above external device connection mode, the external device is connected so that communication can be made, and either the focused noise cancellation mode or the conversation support mode can be selected and operated together. In the above external device connection mode, the communication unit operates to receive sound from the external device and output it through the speaker. Hearing assistance devices.

8. In claim 1, Further comprising a necklace means for connecting the two earplug bodies so that they can be worn around the user's neck. Hearing Assistive Devices 9. In claim 8, The above necklace means comprises two connecting parts that are detachably connected to each of the two earplug bodies, and a connecting string connecting the two connecting parts. Hearing assistance devices.

10. In claim 1, Further comprising an ear cap detachably connected to the earplug body, Hearing assistance devices.

Images