Smart hearing device, artificial intelligence hearing system and method for distinguishing natural or non-natural language

Abstract

The present invention relates to a smart hearing device that analyzes sound data and provides control parameters and feedback for a determined natural or non-natural language, and includes a receiving unit that receives sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side, a determining unit that compares the digital flow of the sound data with a pre-stored graph pattern to determine whether the sound data is a natural or non-natural language, a processing unit that matches similar data for the determined natural or non-natural language based on a database including a natural language domain and a non-natural language domain, and a providing unit that sets control parameters for the natural or non-natural language identified by the matched similar data and provides the converted sound on one side to a user.

Description

[Technical field]

[0001] The present invention relates to a smart hearing device, an artificial intelligence hearing system, and a method thereof for distinguishing between natural and non-natural words, and more particularly to a technique for providing control parameters and feedback for natural or non-natural words determined by analyzing sound data. [Background technology]

[0002] Thanks to the rapid development of medical and engineering technology, patients who previously could not see significant results from wearing hearing aids can now experience significant improvements in their hearing by selecting and wearing the appropriate hearing aid.

[0003] Hearing aids are cutting-edge medical devices that are always attached to the body. Hearing aids require ongoing maintenance as hearing changes, and after-sales service for any damage caused by moisture or foreign objects in the ear. Therefore, hearing aids can be said to be one of the most important technologies in biomedical engineering.

[0004] Early hearing aids were trumpet-shaped sound amplifiers, but nowadays, electronic hearing aids are used that help amplify sound. There are also bone-conduction hearing aids that are attached to the mastoid, but most of these are air-conduction type, in which sound waves are received by a microphone, converted into electrical vibrations, amplified, and then converted back into sound waves so that they can be heard through earphones.

[0005] In recent years, there has been research into more powerful hearing aid specific processors. The hearing aid processor has a processing speed and memory that is more than twice as fast as existing processors, and is made up of miniaturized chips and components constructed using advanced nanotechnology.

[0006] However, conventional hearing aid technology is limited in that it is set up based on the hearing data of the hearing-impaired person (hereinafter referred to as the "user") and cannot adapt to the user's surrounding noise in real time.

[0007] In addition, conventional hearing aid technology distinguishes between voices and external noise by inferring voices through classification of sound ranges, but the performance is not excellent and the technology is unable to distinguish between natural and non-natural speech. Furthermore, the human voice has a fairly wide frequency range of 500 to 5000 Hz, but conventional hearing aid technology recognizes certain voice frequency ranges as external noise and is therefore unable to properly realize voice extraction and noise reduction functions. Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention aims to distinguish between a user's voice and noise by dividing the user into natural and non-natural languages, and to provide effective hearing aid performance by using different parameters and feedback.

[0009] Another object of the present invention is to enable a user to three-dimensionally recognize environmental changes and noise changes due to the directionality of sound on the left and right sides by using a first smart hearing device and a second smart hearing device worn on the right and left sides of the user, respectively. [Means for solving the problem]

[0010] A smart hearing device according to an embodiment of the present invention includes a receiving unit that receives sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side, a determining unit that compares a digital flow of the sound data with a pre-stored graph pattern to determine whether the sound data is a natural language or a non-natural language, a processing unit that matches similar data for the determined natural language or non-natural language based on a database including a natural language domain and a non-natural language domain, and a providing unit that sets control parameters to the natural language or non-natural language identified by the matched similar data and provides the converted one-side sound to a user.

[0011] In addition, the smart hearing device according to an embodiment of the present invention may further include a feedback transmitter that provides feedback in response to the identified natural or non-natural language based on the matched similar data.

[0012] The receiving unit may sense the voice signal and the noise signal from the first microphone and the second microphone formed at different positions, and may digitally convert the voice signal and the noise signal to receive the sound data.

[0013] The determination unit may compare the digital flow of the digitally converted sound data with pre-stored graph patterns of volume versus time graphs and volume versus frequency graphs, and determine whether the graph values ​​are natural or non-natural words that show a specific pattern.

[0014] Based on the database, the processing unit may match similar data of words and sentences that are similar to the sound data determined to be natural language, or may match similar data of environmental noise, noise, and new sounds that are similar to the sound data determined to be non-natural language.

[0015] The providing unit may provide one-sided sound in a personalized manner by setting at least one of the control parameters of an amplification value change due to an environmental change, a volume adjustment, and a frequency adjustment based on the natural language or non-natural language identified by the user's hearing data and the matched similar data.

[0016] The providing unit may adjust the balance of at least one of an amplification value change, a volume adjustment, and a frequency adjustment by setting the control parameters to the sound data of the digital signals received from the first microphone and the second microphone, and convert the digital signal for the adjusted signal into an analog signal to provide one-side sound to the user.

[0017] When the processing unit matches the similar data of words and sentences similar to the sound data determined by the processing unit as a natural language or a non-natural language, the feedback transmission unit may provide feedback of words and sentences in response to the similar data.

[0018] The feedback transmission unit may extract feedback of words and sentences in response to the user's voice question and transmit the feedback as a voice message when the similar data of similar words and sentences is matched to the sound data based on artificial intelligence (AI).

[0019] An artificial intelligence hearing system according to an embodiment of the present invention includes a first smart hearing device that determines whether a first sound data including a voice signal and a noise signal is a natural word or a non-natural word, and provides a sound on one side by setting a first control parameter based on matching similar data for the determined natural word or non-natural word, a second smart hearing device that determines whether a second sound data including a voice signal and a noise signal is received from a third microphone and a fourth microphone that are formed on the other side, and provides a sound on the other side by setting a second control parameter based on matching the similar data for the determined natural word or non-natural word, and a mobile device that controls the first smart hearing device and the second smart hearing device.

[0020] Each of the first smart hearing device and the second smart hearing device may include the first microphone and the third microphone located near the user's mouth, and the second microphone and the fourth microphone located at a distance relative to the user's mouth.

[0021] The first smart hearing device and the second smart hearing device may each provide a user with personalized one-side sound and other-side sound, respectively, allowing the user to stereoscopically recognize environmental changes and noise changes due to the directionality of sound from the left and right sides.

[0022] The first smart hearing device and the second smart hearing device may set the first control parameter and the second control parameter to different parameter values ​​based on the left and right hearing data of a user, respectively.

[0023] The mobile device may control one or more of the power on / off, signal collection, and control parameter settings of each of the first smart hearing device and the second smart hearing device according to a user's selection input.

[0024] An operating method of a smart hearing device for distinguishing between natural and non-natural words according to an embodiment of the present invention, comprising the steps of receiving sound data of voice signals and noise signals from a first microphone and a second microphone formed on one side, determining whether the sound data is a natural or non-natural word by comparing a digital flow of the sound data with a pre-stored graph pattern, matching similar data for the determined natural or non-natural word based on a database including a natural language domain and a non-natural language domain, and setting a control parameter to the natural or non-natural word identified by the matched similar data, and providing the converted one-side sound to a user.

[0025] In addition, the method of operating a smart hearing device according to an embodiment of the present invention may further include providing feedback in response to the identified natural or non-natural words according to the matched similar data. Effect of the Invention

[0026] According to an embodiment of the present invention, it is possible to distinguish between a user's voice and noise by dividing the user into natural language and non-natural language, and provide effective hearing aid performance by using different parameters and feedback.

[0027] In addition, according to an embodiment of the present invention, by using a first smart hearing device and a second smart hearing device worn on the right and left sides of the user, respectively, a personalized hearing aid service can be provided in response to environmental changes and noise changes due to sound directionality on the left and right sides, thereby increasing the convenience of using the hearing aid. [Brief description of the drawings]

[0028] [Figure 1] FIG. 2 is a block diagram showing a detailed configuration of a smart hearing device according to an embodiment of the present invention.

[0029] [Diagram 2] 4 is a flow chart illustrating a method of operation of a smart hearing device in accordance with an embodiment of the present invention.

[0030] [Figure 3a] FIG. 3a is a diagram for explaining an example of providing feedback by classifying natural language or non-natural language according to an embodiment of the present invention. [Figure 3b] FIG. 3b is a diagram for explaining an example of providing feedback by classifying natural language or non-natural language according to an embodiment of the present invention.

[0031] [Figure 4a] FIG. 4a illustrates an example of a smart hearing device according to an embodiment of the present invention. [Figure 4b] FIG. 4b shows an example of a smart hearing device according to an embodiment of the present invention.

[0032] [Diagram 5] FIG. 1 is a diagram showing an application example of a smart hearing device in an embodiment of the present invention. [Figure 6a] FIG. 6a is a diagram illustrating an application example of a smart hearing device in an embodiment of the present invention. [Figure 6b] FIG. 6b is a diagram illustrating an application example of a smart hearing device in an embodiment of the present invention.

[0033] [Figure 7] 1 is a diagram showing the configuration of an artificial intelligence hearing system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the invention should not be restricted or limited by the embodiments. The same reference symbols in the various drawings indicate the same elements.

[0035] The terminology used in this specification is used to describe the embodiments of the present invention, and may vary depending on the intentions of the viewer or operator, or the conventions of the field to which the present invention pertains. Therefore, the definitions for such terms must be based on the overall content of this specification.

[0036]

[0037] The present invention relates to a smart hearing device, an artificial intelligence hearing system, and a technology related to the method for distinguishing between natural and non-natural languages, and its gist is that it analyzes sound data of voice signals and noise signals received from a smart hearing device worn on the user's ear to determine whether the language is natural or non-natural, and provides control parameter setting and feedback by matching similar data based on the judgment results.

[0038] Here, the smart hearing device according to an embodiment of the present invention is a hearing aid that provides amplified sound so that a user with poor hearing ability can hear it.

[0039] Hereinafter, a smart hearing device, an artificial intelligence hearing system, and a method thereof for distinguishing natural or non-natural language according to an embodiment of the present invention will be described in detail with reference to Figures 1 to 7, which controls at least one of the amplification value, volume, and frequency of a signal depending on the determined natural or non-natural language and provides feedback thereto, thereby enhancing the convenience of personalized hearing aid services for users.

[0040]

[0041] FIG. 1 is a block diagram showing a detailed configuration of a smart hearing device in an embodiment of the present invention.

[0042] Referring to FIG. 1, a smart hearing device according to an embodiment of the present invention provides control parameters and feedback for natural or non-natural words determined by analyzing sound data.

[0043] A smart hearing device 100 according to an embodiment of the present invention described below is worn on the left or right ear of a user and may be a first smart hearing device or a second smart hearing device. The first smart hearing device and the second smart hearing device are as shown in FIG. 1 and are characterized in that they include the same detailed configuration as described below.

[0044] To this end, the smart hearing device 100 according to the embodiment of the present invention includes a receiving unit 110, a determining unit 120, a processing unit 130, and a providing unit 140. In addition, the smart hearing device 100 according to the embodiment of the present invention may further include a feedback transmission unit 150 and a database 160.

[0045] The receiving unit 110 receives sound data of a voice signal and a noise signal from a first microphone 101 and a second microphone 102 formed on one side.

[0046] The receiving unit 110 may sense a voice signal and a noise signal from a first microphone 101 and a second microphone 102 formed at different positions, and may digitally convert the voice signal and the noise signal to receive the signal as sound data.

[0047] In this case, the first microphone 101 may receive a voice signal of the user, and the second microphone 102 may receive a noise signal around the user. More specifically, the first microphone 101 and the second microphone 102 are positioned at different distances from each other with respect to the user's mouth, with the first microphone 101 being stably positioned near the user's mouth and mainly receiving audio signals which are the user's voice, and the second microphone 102 being positioned as far away as possible from the user's mouth compared to the first microphone 101 and mainly receiving noise signals which are ambient noise.

[0048] The first microphone 101 and the second microphone 102 according to the embodiment of the present invention are arranged at different positions in the smart hearing device 100, but the directions of the apertures (or holes) of the first microphone 101 and the second microphone 102 are the same in order to sense uniform voice signals and noise signals and thereby perform appropriate noise elimination. Here, the appropriate noise may refer to noise and numerical values ​​other than the voice signal and noise signal collected at the microphone position.

[0049] As a result, the receiver 110 receives the voice signal and noise signal sensed by the first microphone 101 and the second microphone 102 and converts them into electrical signals, that is, digitally converts them and receives them as sound data.

[0050] The determination unit 120 compares the digital flow of the sound data with a pre-stored graph pattern to determine whether the sound data is a natural or non-natural language.

[0051] The judgment unit 120 may compare the digital flow of digitally converted sound data with pre-stored graph patterns of volume graphs versus time and volume graphs versus frequency based on artificial intelligence (AI), and determine whether the graph values ​​indicate a natural or non-natural word that shows a particular pattern.

[0052] In an embodiment of the present invention, natural language refers to words or sentences such as human conversations or user questions, and non-natural language refers to noises such as noises at a construction site, automobile sounds, sounds temporarily generated from objects, and sounds unfamiliar to humans (or new sounds). Each natural and non-natural language shows a different specific pattern in the time-volume graph and the frequency-volume graph, and the specific patterns according to the graphs may be stored and maintained in the database 160 . For example, user dialogue has higher and lower tones and is more sustained than noise generated at a construction site. This results in a certain pattern in the time-volume graph and the frequency-volume graph for the text of a user's dialogue, and a certain pattern in the time-volume graph and the frequency-volume graph for the noise generated at a construction site.

[0053] The determination unit 120 may compare the digital flow of the digitally converted sound data with graph patterns for natural and non-natural words stored and maintained in the database 160 to determine whether the natural or non-natural word shows a particular pattern. For example, the determination unit 120 may compare and analyze the digitally converted sound data to determine whether it is a natural language or a non-natural language.

[0054] The processing unit 130 matches similar data for the determined natural or non-natural language based on a database 160 including natural language and non-natural language domains.

[0055] When the judgment unit 120 judges the sound data to be a natural language or a non-natural language, the processing unit 130 may match the sound data judged to be a natural language with similar data of words and sentences similar to the sound data judged to be a natural language based on the database 160, or may match the sound data judged to be a non-natural language with similar data of environmental noise, noise, and new sounds similar to the sound data. More specifically, the database 160 may include a natural language domain including words or sentences related to natural language and a non-natural language domain including information related to noise and unfamiliar sounds, and when the judgment unit 120 judges the sound data to be natural language, the processing unit 130 may match similar data of words or sentences similar to the sound data in the natural language domain based on the database 160.

[0056] For example, if the determination unit 120 determines that the noise of a construction site or the sound of an automobile is a non-natural language, the processing unit 130 may match similar data as the noise of a construction site or the sound of an automobile. Furthermore, when the determining unit 120 determines that an unfamiliar sound is a non-natural word, the processing unit 130 may match the similar data as a new sound or a new noise.

[0057] As another example, if the determination unit 120 determines that the voices of the user and the people around him / her are natural language, the processing unit 130 may match similar data such as "hello" and "long time no see." In addition, if the determination unit 120 determines that the user's question is in natural language, the processing unit 130 may match similar data such as "What's the weather like today?" or "I'm hungry" through a related search in the database 160.

[0058] The providing unit 140 sets a control parameter to the natural language or the non-natural language specified by the matched similar data, and provides the converted one-side sound to the user.

[0059] At this time, the smart hearing device 100 according to the embodiment of the present invention may basically include hearing data (Personal Hearing Profile) of a user who uses a hearing aid. For example, the providing unit 140 may include the user's hearing data including the user's preferred volume and frequency, and the amplification value, volume, and frequency range at which the user does not perceive a sense of foreignness. Depending on the embodiment, the above-mentioned data may be received from a mobile device 500 or an external server 600 that interfaces with the smart hearing device 100 according to an embodiment of the present invention, and may be stored and maintained in the mobile device 500 or the external server 600.

[0060] However, the hearing data should not be limited to items or numerical values ​​such as amplification value, volume, and frequency. For example, the hearing data may further include the user's preferences and numerical values ​​for at least one of nonlinear compression information that amplifies soft sounds and reduces loud sounds, directional information that accurately captures the direction from which a sound is coming, feedback information that amplifies sounds received by a microphone to help them be heard without other noise, and noise cancellation information that reduces noise.

[0061] The providing unit 140 may set at least one or more control parameters of an amplification value change due to an environmental change, a volume adjustment, and a frequency adjustment based on a natural or non-natural word identified by similar data matched with the user's hearing data, and provide one-sided sound through personalization. In this case, the providing unit 140 applies different control parameter values ​​to natural language and non-natural language. For example, even if the same control parameter for increasing the volume is applied, the providing unit 140 may increase the control parameter value applied to natural words more than that applied to non-natural words.

[0062] More specifically, the providing unit 140 may set control parameters for the sound data of the digital signals received from the first microphone 101 and the second microphone 102, adjust the balance of at least one of an amplification value change, a volume adjustment, and a frequency adjustment, and convert the digital signal for the adjusted signal into an analog signal to provide one-sided sound to the user.

[0063] For example, at least one of the amplification value, volume, and frequency of the sound data received from the first microphone 101 and the second microphone 102 may deviate from a standard range preset by the user or preferred by the user. This can be said to occur due to one of the following factors: a change in the environment in which the user is located, a change in the user's voice, a mechanical error, and a pitch change due to natural or non-natural language. Thus, the providing unit 140 may adjust the balance of at least one of the amplification value, volume, and frequency of the voice signal and the noise signal based on similarity data for the natural language or non-natural language matched by the processing unit 130, and may convert the digital signal based on the adjusted balance into an analog signal (acoustic energy) and provide it to the user as sound.

[0064] For example, when the processing unit 130 matches similar data as noise from a construction site or noise from an automobile, the providing unit 140 may adjust the balance to lower the volume of the noise and provide it to the user.

[0065] As another example, when the processing unit 130 matches the voices of the user and people in the vicinity with similar data such as “Hello” or “It’s been a while,” the providing unit 140 may adjust the balance to increase the amplification value, volume, and frequency of the matched similar data over the surrounding noise, and provide the information so that it is accurately conveyed to the user.

[0066] In other words, the smart hearing device 100 according to an embodiment of the present invention analyzes sound data including voice signals and noise signals received from the first microphone 101 and the second microphone 102 in response to changes in the user's environment to determine whether the sound data is natural or non-natural, and matches similar data for the natural or non-natural words to set control parameters for the sound data. This makes it possible to provide optimal hearing aid services for changing situations without the user having to adjust the volume or frequency individually, thereby improving the convenience of using the hearing aid.

[0067] The feedback transmission unit 150 of the smart hearing device 100 according to an embodiment of the present invention may provide feedback as a response to the identified natural language according to the matched similar data.

[0068] When the sound data determined by the processing unit 130 to be a natural language or a non-natural language is matched with similar data of words and sentences similar to the sound data, the feedback transmitting unit 150 may provide feedback of the words and sentences in response to the similar data.

[0069] When similar data of similar words and sentences to the sound data is matched based on artificial intelligence (AI), the feedback transmission unit 150 may extract feedback of words and sentences in response to the user's voice question and transmit it by voice.

[0070] As an example, when the judgment unit 120 judges the noise of a construction site or the sound of an automobile as a non-natural word and the processing unit 130 matches similar data of the noise of a construction site or the noise of an automobile, the balance of the sound data corresponding to the non-natural word may be adjusted according to the control parameters. Thereafter, the feedback transmitting unit 150 may provide the user with feedback regarding the sound data corresponding to the non-natural language. For example, when similar data is matched as automobile noise, the feedback transmission unit 150 may provide a voice message notification such as "Be careful" to the user. As another example, when similar data is matched as a new noise that is unfamiliar to the user, the feedback transmission unit 150 may provide a voice message notification such as "This is a new sound" or "This is a new environment" to the user.

[0071] As another example, when the judgment unit 120 judges a user's question as natural language and the processing unit 130 matches similar data such as "How's the weather today?" or "I'm hungry," the balance of the sound data corresponding to the natural language may be adjusted according to the control parameters. Thereafter, the feedback transmitting unit 150 may provide the user with feedback on the sound data corresponding to the natural language. For example, when similar data of "What's the weather like today?" is matched, the feedback transmission unit 150 may provide a voice message notification such as "The weather is fine today" to the user. As another example, when similar data such as "I'm hungry" is matched, the feedback transmission unit 150 may provide a voice message notification such as "The nearest restaurant from here is OOO" to the user.

[0072] The smart hearing device 100 according to an embodiment of the present invention may further include a communication unit (not shown) for transmitting sound data, similar data, control parameters, and information related to feedback to the mobile device or an external server, and for receiving user input from the mobile device.

[0073] For example, the communication unit may transmit sound data, similar data, control parameters, and information related to feedback to a mobile device carried by a user through any one of short-range wireless communication modules such as Bluetooth, Wi-Fi (Wireless Fidelity), Zigbee, and BLE (Bluetooth Low Energy), and may receive a control command for turning the power on / off or controlling the control parameters of the smart hearing device based on a selection input by the user from the mobile device.

[0074] Depending on the embodiment, the communication unit of the smart hearing device 100 according to the embodiment of the present invention may communicate via wireless communication with not only a short-range wireless communication module, but also wireless networks such as cellular telephone networks, wireless local area networks (LANs) and / or metropolitan area networks (MANs), networks such as intranets and / or the Internet known as the World Wide Web (WWW), and other devices.

[0075] Such wireless communications may include, but are not limited to, Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, Long Term Evolution (LTE), Zigbee, Z-wave, Bluetooth Low Energy (BLE), Beacon, Internet Message Access Protocol (IMAP) and / or email protocols such as Post Office Protocol (POP), eXtensible Messaging and Presence Protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Leveraging Extensions (IMPS), and / or other protocols for mobile communications. The communication protocols may include instant messaging such as Messaging and Presence Service (SMS), Short Message Service (SMS), LoRa, or communications protocols not yet developed as of the filing date of this application. However, multiple communication standards, protocols, and technologies may be used for wireless communication, including but not limited to those described above.

[0076]

[0077] FIG. 2 is a flow chart illustrating a method of operation of a smart hearing device in accordance with an embodiment of the present invention.

[0078] The method of operation of FIG. 2 is performed by the smart hearing device 100 according to the embodiment of the present invention of FIG.

[0079] Referring to FIG. 2, in step 210, sound data of a voice signal and a noise signal are received from a first microphone and a second microphone formed on one side.

[0080] In step 210, a voice signal and a noise signal may be sensed from a first microphone and a second microphone formed at different positions, and in step 220, the voice signal and the noise signal may be digitally converted and received as sound data.

[0081] In step 230, the digital flow of sound data and the pre-stored graph pattern are used to determine natural or non-natural language for the sound data.

[0082] Stage 230 may compare the digital flow of digitally converted sound data based on artificial intelligence (AI) with pre-stored graph patterns of volume versus time graphs and volume versus frequency graphs, and determine natural or non-natural words that show certain patterns in the graph values.

[0083] In step 230, the sound data is classified as a natural or non-natural word, and in steps 241 and 251, similar data is matched to the determined natural or non-natural word based on a database including natural and non-natural language domains. For example, step 241 may match similar data to construction site noises or automobile noises if construction site noises or automobile noises are determined as non-natural words by step 230 . In addition, in step 251, if the voices of the user and people around the user are determined to be natural language in step 230, similar data such as "Hello" or "It's been a while" may be matched, and if the user's question is determined to be natural language in step 230, similar data such as "How's the weather today?" or "I'm hungry" may be matched by a related search in a database.

[0084] Thereafter, in steps 242 and 252, control parameters are set to the natural or non-natural language specified by the matched similar data, and the converted one-sided sound is provided to the user.

[0085] Steps 242 and 252 may set at least one or more control parameters of an amplification value change due to an environmental change, a volume adjustment, and a frequency adjustment based on the natural or non-natural words identified by the similar data matched with the user's hearing data, thereby providing one-sided sound through personalization.

[0086] For example, if similar data is matched to noise from a construction site or automobile noise in step 241, step 242 may provide the user with a lowered volume balance adjustment for the noise. Also, in step 251, if the voices of the user and people in the vicinity are matched to similar data such as “hello” or “long time no see,” step 252 may adjust the balance to boost the amplification value, volume, and frequency of the matched similar data over the surrounding noise, so that information can be accurately conveyed to the user.

[0087] At this time, steps 242 and 252 are characterized by applying different control parameter values ​​to natural words and non-natural words, respectively. More specifically, even if similar volume-increasing control parameters are applied, the control parameters applied to natural words may be higher than the control parameters applied to non-natural words.

[0088] Thereafter, in steps 243 and 253, feedback may be provided in response to the identified natural or non-natural words from the matched similar data.

[0089] Steps 243 and 253 may extract and transmit feedback of words and sentences in voice as a response to a user's question when similar data of similar words and sentences to the sound data is matched based on artificial intelligence (AI).

[0090] For example, if the similar data is matched as an automobile noise, step 243 may provide a voice message notification to the user such as "Be careful," and if the similar data is matched as a new noise that is unfamiliar to the user, step 243 may provide a voice message notification to the user such as "New sound" or "New environment." Also, if similar data of "What's the weather like today?" is matched, step 253 may provide a voice message notification such as "The weather is sunny today" to the user, and if similar data such as "I'm hungry" is matched, step 253 may provide a voice message notification such as "The nearest restaurant from here is OOO" to the user.

[0091] In the non-natural language domains of steps 241, 242 and 243, the operating method of a smart hearing device according to an embodiment of the present invention may set control parameters to provide a sound suitable for the user for noise, but provide a voice message notification such as "be careful" or "new sound" as feedback for threatening or new sounds.

[0092] Similarly, in the natural language domain of steps 251, 252, and 253, the operating method of the smart hearing device according to an embodiment of the present invention sets control parameters for the sounds of conversation between users to provide sounds suitable for the users, and may provide corresponding responses as feedback to questions posed by the users. In this case, the method of operating the smart hearing device according to the embodiment of the present invention may extract a response appropriate to the user's question in natural language based on artificial intelligence (AI).

[0093]

[0094] 3a and 3b are diagrams illustrating an example of providing feedback by classifying natural or non-natural language according to an embodiment of the present invention.

[0095] To explain in more detail, Figure 3a is a diagram for explaining an example of determining natural language and providing feedback in an embodiment of the present invention, and Figure 3b is a diagram for explaining an example of determining non-natural language and providing feedback in an embodiment of the present invention.

[0096] As shown in FIG. 3a, it is assumed that the smart hearing device according to the embodiment of the present invention detects sound data of "hello" from the first microphone and the second microphone formed on one side. In this case, the sound data of "hello" shows a specific pattern in the time-volume graph and frequency-volume graph as shown in FIG. 3a, and the smart hearing device according to an embodiment of the present invention may compare the digital flow for the sound data with the graph pattern pre-stored in the database and determine the sound data as natural language.

[0097] After determining the natural language, the smart hearing device according to an embodiment of the present invention matches similarity data to the natural language. For example, a smart hearing device according to an embodiment of the present invention may attempt to match similar data close to the detected "hello," and may match similar data of "hello" that is similar to the sound data of "hello" from data such as "hello," "good afternoon," and "good afternoon."

[0098] By matching similar data, a smart hearing device according to an embodiment of the present invention may set at least one or more control parameters, including amplification change, volume adjustment, and frequency adjustment, in natural language, and extract feedback in response to the "hello" sound data and provide it to the user in the form of sound. For example, a smart hearing device according to an embodiment of the present invention may provide a user with voice message notifications such as "Yes, hello," "It's been a while," or "How can we help you?" as feedback to the sound data of "hello."

[0099] As shown in FIG. 3b, it is assumed that the smart hearing device according to the embodiment of the present invention detects sound data of 'cough cough' from the first and second microphones formed on one side. In this case, the sound data of "cough cough" shows a specific pattern in the time-volume graph and frequency-volume graph as shown in Figure 3b, and the smart hearing device according to an embodiment of the present invention compares the digital flow for the sound data with the graph pattern pre-stored in the database and determines the sound data as a non-natural word. As shown in Figures 3a and 3b, it has been confirmed that natural and non-natural languages ​​show different graph patterns in the time-volume graph and frequency-volume graph, so a smart hearing device according to an embodiment of the present invention distinguishes and determines whether a language is natural or non-natural based on the difference in such graph patterns.

[0100] After determining the non-natural words, the smart hearing device according to an embodiment of the present invention matches similarity data for the non-natural words. For example, a smart hearing device according to an embodiment of the present invention may attempt to match data similar to the detected "cough cough," and may match similar data of "cough cough" that is similar to the sound data of "cough cough" from data such as "cough," "cough," and "cough cough."

[0101] By matching similar data, a smart hearing device according to an embodiment of the present invention may set at least one or more control parameters, including amplification change, volume adjustment, and frequency adjustment, to the non-natural words, and extract feedback in response to the "cough cough" sound data and provide it to the user in the form of sound. For example, a smart hearing device according to an embodiment of the present invention may provide a user with voice message notifications such as "Here's where to find a nearby pharmacy," "You seem to have a cold," "Let's have some hot tea," and "Are you okay?" as feedback to the "cough cough" sound data.

[0102]

[0103] 4a and 4b show an example of a smart hearing device according to an embodiment of the present invention.

[0104] As shown in Figures 4a and 4b, smart hearing devices according to embodiments of the present invention may come in a pair to be worn on a user's left and right ears, respectively.

[0105] The first smart hearing device 300 and the second smart hearing device 400 described below include the same functions as the smart hearing devices according to the embodiments of the present invention described in Figures 1 to 3b.

[0106] Referring to FIG. 4a, a first smart hearing device 300 according to an embodiment of the present invention includes a first microphone 301, a second microphone 302 and an on / off switch 303, and a second smart hearing device 400 includes a third microphone 401, a fourth microphone 402 and an on / off switch 403. In the drawings, the first smart hearing device 300 is shown worn on the user's left ear and the second smart hearing device 400 is shown worn on the user's right ear, but the position and manner in which the smart hearing devices are worn should not be limited to this.

[0107] In this case, the first microphone 301 and the third microphone 401 may be positioned adjacent to the user's mouth in order to receive audio signals, which are mainly the user's voice, and are characterized by being located below the on / off switches 303, 403 and being relatively closer to the user's mouth than the second microphone 302 and the fourth microphone 402.

[0108] In addition, the second microphone 302 and the fourth microphone 402 may be located as far away as possible from the user's mouth in order to receive noise signals, which are mainly ambient noise depending on the user's position, and are characterized by being located above the on / off switches 303, 403 and relatively far away from the user's mouth compared to the first microphone 301 and the third microphone 401.

[0109] Furthermore, the directions of the apertures (or holes) of the first microphone 301, the second microphone 302, the third microphone 401 and the fourth microphone 402 are characterized by being the same. This is to collect uniform speech and noise signals and remove the appropriate noise from them.

[0110] As shown in FIG. 4a, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention include two microphones 301, 302 and 401, 402 positioned at different positions, and in terms of software, the first microphone 301 and the third microphone 401 are set as the main microphones, and the second microphone 302 and the fourth microphone 402 are used as secondary input sources, thereby enabling different voice signals and noise signals to be collected uniformly.

[0111] At this time, the first microphone 301 of the first smart hearing device 300 and the third microphone 401 of the second smart hearing device 400 are paired with each other, and the second microphone 302 of the first smart hearing device 300 and the fourth microphone 402 of the second smart hearing device 400 are paired with each other, and the microphone to be paired with any one of the microphones may also be automatically set according to the settings applied to any one of the microphones.

[0112] For example, when the volume of the first microphone 301 is increased to a certain value by the control parameter, the volume of the paired third microphone 401 may also be automatically adjusted to be increased to a certain value. As another example, when the second microphone 302 of the first smart hearing device 300 is turned on, the fourth microphone 402 of the paired second smart hearing device 400 may also be automatically turned on.

[0113] Referring to FIG. 4a, the first smart hearing device 300 and the second smart hearing device 400 include on / off switches 303, 403, which serve to turn the power of the first smart hearing device 300 and the second smart hearing device 400 on or off. For example, when a user touches, pushes, or presses and holds the on / off switches 303, 403 in the form of switches, the first smart hearing device 300 and the second smart hearing device 400 may be turned on or off. In this case, when at least one of the first smart hearing device 300 and the second smart hearing device 400 is turned on or off, the other paired smart hearing device may also be turned on or off in the same manner.

[0114] Referring to FIG. 4b, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention include a charging module 305, 405 and a speaker 304, 404.

[0115] The first smart hearing device 300 and the second smart hearing device 400 according to the embodiment of the present invention may include charging modules (Charging Terminals) 305, 405 as rechargeable devices.

[0116] For example, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention may include a rechargeable Li-ion Polymer Battery and a mobile device battery meter, and the battery may be charged through the charging module 305, 405.

[0117] Furthermore, the first smart hearing device 300 and the second smart hearing device 400 according to the embodiment of the present invention may provide sound converted from a digital signal to an analog signal (acoustic energy) through the speakers 304, 404.

[0118] For example, the first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention may set first and second control parameters based on information related to the analysis results for the voice signals and noise signals collected from the first microphone 301, the second microphone 302, the third microphone 401, and the fourth microphone 402, convert the digital signals in which the balance of at least one of the amplification value change, volume adjustment, and frequency adjustment has been adjusted, into analog signals, and provide them to the user as sound through the speakers 304, 404.

[0119]

[0120] 5, 6a and 6b are diagrams showing application examples of a smart hearing device according to an embodiment of the present invention.

[0121] More specifically, FIG. 5 shows an example of a user wearing a smart hearing device according to an embodiment of the present invention, viewed from the front. FIG. 6a shows an example of a user wearing a first smart hearing device according to an embodiment of the present invention viewed from the left side, and FIG. 6b shows an example of a user wearing a second smart hearing device according to an embodiment of the present invention viewed from the right side.

[0122] Referring to FIG. 5, a user 10 wears a first smart hearing device 300 on the left ear and a second smart hearing device 400 on the right ear. By wearing both the first smart hearing device 300 and the second smart hearing device 400, the user 10 can perceive environmental changes and noise changes due to the directionality of sound from the left and right sides in a more three-dimensional manner, and can receive personalized hearing aid services.

[0123] Referring to FIG. 6a, a first smart hearing device 300 according to an embodiment of the present invention is worn on the left ear of a user 10, and a first microphone 301 and a second microphone 302 may be located at different distances from each other relative to the user's mouth.

[0124] For example, the first microphone 301 may be located closer to the user's mouth than the second microphone 302, and may receive audio signals that are mainly the user's voice. On the other hand, the second microphone 302 is positioned as far away from the user's mouth as possible compared to the first microphone 301, and may receive noise signals that are mainly ambient noise depending on the user's position.

[0125] At this time, as shown in FIG. 6a, it can be confirmed whether the first microphone 301 and the second microphone 302 are located near or far from the user's mouth based on the on / off switch 303.

[0126] The first microphone 301 and the second microphone 302 are included in different positions in the first smart hearing device 300 according to an embodiment of the present invention, but the directions of the apertures (or inlets) of the first microphone 301 and the second microphone 302 are the same in order to collect uniform voice signals and noise signals, thereby eliminating noise appropriately.

[0127] Referring to FIG. 6b, a second smart hearing device 400 according to an embodiment of the present invention is worn on the right ear of a user 10, and a third microphone 401 and a fourth microphone 402 may be located at different distances from each other based on the user's mouth.

[0128] For example, the third microphone 401 may be located closer to the user's mouth than the fourth microphone 402, and may receive audio signals that are mainly the user's voice. On the other hand, the fourth microphone 402 is positioned as far away from the user's mouth as possible compared to the third microphone 401, and may receive noise signals that are mainly ambient noise depending on the user's position.

[0129] At this time, as shown in FIG. 6b, it may be confirmed that the third microphone 401 and the fourth microphone 402 are positioned close to or far from the user's mouth based on the on / off switch 403.

[0130] The third microphone 401 and the fourth microphone 402 are included in different positions in the second smart hearing device 400 according to an embodiment of the present invention, but the directions of the port (or inlet) of the third microphone 401 and the fourth microphone 402 are the same in order to collect uniform voice signals and noise signals, thereby eliminating noise appropriately.

[0131]

[0132] FIG. 7 is a diagram showing the configuration of an artificial intelligence hearing system in an embodiment of the present invention.

[0133] Referring to FIG. 7, an artificial intelligence hearing system according to an embodiment of the present invention includes a first smart hearing device 300 formed on one side and a second smart hearing device 400 formed on the other side, and a mobile device 500 that controls the first smart hearing device 300 and the second smart hearing device 400. In addition, the artificial intelligence hearing system according to an embodiment of the present invention may further include an external server 600.

[0134] The first smart hearing device 300 determines whether first sound data including a voice signal and a noise signal received from a first microphone and a second microphone formed on one side is a natural word or a non-natural word, and provides sound from one side by setting a first control parameter based on matching similar data for the determined natural word or non-natural word. In addition, the second smart hearing device 400 determines whether the second sound data, including the voice signal and the noise signal, received from the third microphone and the fourth microphone formed on the other side is a natural word or a non-natural word, and provides sound from the other side by setting a second control parameter based on matching the similar data for the determined natural word or non-natural word.

[0135] The first smart hearing device 300 and the second smart hearing device 400 may include a first microphone and a third microphone located near the user's mouth, and may also include a second microphone and a fourth microphone located at a distance from the user's mouth. In this case, the first and third microphones may collect the user's voice signals on one side and the other side, and the second and fourth microphones may collect the noise signals on one side and the noise signals on the other side.

[0136] More specifically, the first smart hearing device 300 may be worn on the left ear of a user and may receive a voice signal from the left side of the user through a first microphone positioned near the user's mouth, and may receive a noise signal from the left side through a second microphone positioned at a distance from the user's mouth. In addition, the second smart hearing device 400 may be worn on the right ear of a user and may receive a voice signal of the user on the right side through a third microphone positioned near the user's mouth, and may receive a noise signal from the right side through a fourth microphone positioned at a distance from the user's mouth.

[0137] At this time, the first microphone of the first smart hearing device 300 and the third microphone of the second smart hearing device 400, and the second microphone of the first smart hearing device 300 and the fourth microphone of the second smart hearing device 400 are characterized by being paired with each other, and the microphone to be paired with any one of the microphones may also be automatically set according to the settings applied to any one of the microphones. For example, when the volume of the first microphone is adjusted by the first control parameter, the volume of the paired third microphone may also be automatically adjusted. As another example, when the second microphone of the first smart hearing device 300 is turned on, the fourth microphone of the paired second smart hearing device 400 may also be automatically turned on.

[0138] The first smart hearing device 300 and the second smart hearing device 400 may set first and second control parameters with different parameter values ​​based on the left and right hearing data of the user, respectively.

[0139] As an example, the first smart hearing device 300 and the second smart hearing device 400 may each include hearing data (Personal Hearing Profile) for the left and right sides of a user who uses a hearing aid. For example, because the hearing data for a user's left and right sides may be different, each of the first smart hearing device 300 and the second smart hearing device 400 may include personalized hearing data including the user's preferred volume, specific volumes and specific frequencies that can be perceived, and amplification values, volumes, and frequency ranges that do not produce a sense of foreignness. At this time, the user's hearing data may be stored and maintained in the mobile device 500 and the external server 600.

[0140] However, the hearing data should not be limited to items or numerical values ​​such as amplification value, volume, and frequency. For example, the hearing data may further include the user's preferences and numerical values ​​for at least one of nonlinear compression information that amplifies soft sounds and attenuates loud sounds, directional information that accurately captures the direction from which a sound is coming, feedback information that amplifies sounds received by a microphone to make them easier to hear without other noise, and noise cancellation information that reduces noise.

[0141] The first smart hearing device 300 and the second smart hearing device 400 according to an embodiment of the present invention can provide personalized hearing aid services for the left and right sides by setting at least one of first and second control parameters, including amplification value changes, volume adjustments, and frequency adjustments due to environmental and noise changes, based on result information for the user's hearing data and similar data determined to be natural or non-natural language.

[0142] As an example, the first smart hearing device 300 may analyze sound data of the voice signal and noise signal of the digital signal received from the first microphone and the second microphone to determine whether the signal is a natural or non-natural language, set a first control parameter to similar data matched based on the determination result, adjust the balance of at least one of an amplification value change, a volume adjustment, and a frequency adjustment, convert the adjusted signal and the digital signal for feedback into an analog signal, and transmit it to the user as a left sound.

[0143] As another example, the second smart hearing device 400 may analyze sound data of the voice signal and noise signal of the digital signal received from the third microphone and the fourth microphone to determine whether the language is natural or non-natural, set a second control parameter to similar data matched based on the determination result, adjust the balance of at least one of amplification value change, volume adjustment, and frequency adjustment, convert the adjusted signal and the digital signal for feedback into an analog signal, and transmit it to the user as right sound.

[0144] For example, at least one of the amplification value, volume, and frequency of the sound data received from the first microphone, the second microphone, the third microphone, and the fourth microphone may deviate from a standard range preset or preferred by the user. This can be attributed to one of the following factors: a change in the environment in which the user is located, a change in the user's voice, a mechanical error, and a pitch factor due to natural or non-natural language. As a result, the first smart hearing device 300 and the second smart hearing device 400 may apply the first control parameter and the second parameter to the sound data to adjust the balance of at least one of the amplification value, volume, and frequency for the sound data, and convert the digital signal based on the adjusted balance into an analog signal (acoustic energy) and provide it to the user as sound.

[0145] In other words, the first smart hearing device 300 and the second smart hearing device 400 of the artificial intelligence hearing system according to an embodiment of the present invention analyze sound data including voice signals and noise signals received from the first microphone, the second microphone, the third microphone, and the fourth microphone in accordance with changes in the user's environment and changes in noise, determine the sound data as natural or non-natural language, and set first and second control parameters for the matching similar data, thereby providing optimal hearing aid services for changing situations without the user having to adjust the volume or frequency individually, thereby improving the convenience of using the hearing aids.

[0146] In addition, the first smart hearing device 300 and the second smart hearing device 400 of the artificial intelligence hearing system according to an embodiment of the present invention can provide personalized left and right sounds to the user, thereby enabling the user to three-dimensionally perceive environmental changes and noise changes due to the directionality of the left and right sounds.

[0147] The mobile device 500 controls the first smart hearing device 300 and the second smart hearing device 400 .

[0148] As shown in FIG. 7, the first smart hearing device 300, the second smart hearing device 400, and the mobile device 500 may transmit and receive data through Bluetooth communication, which is a short-range wireless communication module. For example, the mobile device 500 may receive information regarding sound data including voice signals and noise signals, judgment results in which natural or non-natural words are judged, similar data, and feedback from the first smart hearing device 300 and the second smart hearing device 400 via Bluetooth communication.

[0149] After this, the mobile device 500 may transmit the information received from the first smart hearing device 300 and the second smart hearing device 400 to the external server 600 via Ethernet / 3G, 4G, or 5G wireless data communication.

[0150] In this case, the mobile device 500 is a terminal device carried by a user who uses the first smart hearing device 300 and the second smart hearing device 400, and may be at least one of a personal computer (PC), a laptop computer, a smart phone, a tablet, and a wearable computer, and may perform general service operations such as service screen configuration, data input, data transmission / reception, data storage, etc. under the control of a web / mobile site or a dedicated application. Additionally, the mobile device 500 may be an application that is downloaded and installed in the mobile device.

[0151] According to an embodiment, the mobile device 500 may display, through a display unit (not shown), a screen including multiple items located in each of multiple regions, and may display another screen including at least one item associated with a function based on a touch-sensitive surface, sensor, or set of sensors that receives input from a user based on haptic or tactile contact. In addition, the mobile device 500 may receive user selection input through an input unit (not shown) such as a keyboard, a touch display, a dial, a slider switch, a joystick, or a mouse, and may output personalized hearing aid service information through an output unit (not shown) including an audio module, a speaker module, and a vibration module.

[0152] The mobile device 500 may provide a screen for testing the user's hearing in conjunction with each of the first smart hearing device 300 and the second smart hearing device 400, and information related to various reports based thereon. In this case, the report may be a history index or record for personalized hearing assistance services over time.

[0153] The mobile device 500 may also store in a database information regarding sound data including voice signals and noise signals from the first smart hearing device 300 and the second smart hearing device 400, the determination results regarding whether the language is natural or non-natural, similar data, and feedback.

[0154] In addition, the mobile device 500 may turn on or off the power of each of the first smart hearing device 300 and the second smart hearing device 400 according to a user's selection input, and may manually control numerical values ​​such as the amplification value, volume, and frequency of the first smart hearing device 300 and the second smart hearing device 400.

[0155] In addition, the mobile device 500 pairs with serial numbers or device information assigned to each of the first smart hearing device 300 and the second smart hearing device 400, and may perform battery management, loss management, and malfunction management for the first smart hearing device 300 and the second smart hearing device 400.

[0156] The external server 600 may digitize and store user information, hearing data based on the user information, and the appropriate ranges of the user's preferred amplification values, volume, frequency, etc., and may match the first smart hearing device 300, second smart hearing device 400, and mobile device 500 corresponding to the user information to create a database. For example, the external server 600 may create a database of sound data from the first smart hearing device 300 and the second smart hearing device 400 received by the mobile device 500 at preset time intervals, the determination results of whether the language is natural or non-natural, similar data, and information regarding feedback.

[0157]

[0158] The above-described apparatus may be realized by hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processor may also respond by executing software to access, record, manipulate, process, and generate data. For ease of understanding, a single processing device may be described as being used, however, those skilled in the art will appreciate that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0159]

[0160] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure or instruct a processing device to operate as desired, either individually or collectively. The software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave to be interpreted by a processing device or to provide instructions or data to a processing device. The software may be distributed over computer systems connected by a network and may be stored and executed in a distributed fashion. The software and data may be stored on one or more computer readable recording media.

[0161]

[0162] Methods according to embodiments may be embodied in the form of program instructions executable by various computer means and recorded on computer readable media. The computer readable media may include, alone or in combination with the program instructions, data files, data structures, and the like. The program instructions contained in the media may be those specially designed and constructed for the embodiment, or they may be of the type well known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include not only machine code, such as produced by a compiler, but also high level language code that is executed by the computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0163]

[0164] As described above, the embodiment has been described based on the limited embodiment and drawings, but those skilled in the art will be able to make various modifications and variations from the above description. For example, the techniques described may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be substituted or replaced by other components or equivalents, and suitable results may still be achieved.

[0165]

[0166] Therefore, different embodiments are within the scope of the appended claims provided they are equivalent to the claims.

Claims

1. a receiving unit for receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side of the receiving unit; a determination unit for determining whether the sound data is a natural language or a non-natural language by comparing the digital flow of the sound data with a pre-stored graph pattern; a processing unit that performs matching of similar data for the determined natural language or non-natural language based on a database including a natural language domain and a non-natural language domain; a providing unit for providing a user with one side of the sound converted by setting a control parameter to the natural language or the non-natural language specified by the matching similar data; a feedback transmitter that provides feedback in response to the identified natural or non-natural language based on the matched similar data; A smart hearing device for distinguishing natural or non-natural language, comprising:

2. a receiving unit for receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side of the receiving unit; a determination unit for determining whether the sound data is a natural language or a non-natural language by comparing the digital flow of the sound data with a pre-stored graph pattern; a processing unit that performs matching of similar data for the determined natural language or non-natural language based on a database including a natural language domain and a non-natural language domain; a providing unit for providing a user with one side of the sound converted by setting a control parameter to the natural language or the non-natural language specified by the matching similar data, the receiving unit detects the voice signal and the noise signal from the first microphone and the second microphone formed at different positions, digitally converts the voice signal and the noise signal, and receives the voice signal and the noise signal as the sound data; The determination unit compares the digital flow of the digitally converted sound data with a graph pattern of a volume graph versus time and a volume graph versus frequency that are stored in advance, and determines whether the graph value indicates a natural or non-natural word that shows a specific pattern. A smart hearing device that distinguishes between natural and non-natural language.

3. The smart hearing device for classifying natural or non-natural language as described in claim 1, characterized in that the processing unit matches the sound data determined to be natural language with similar data of similar words and sentences based on the database, or matches the sound data determined to be non-natural language with similar data of environmental noise, noise, and new sounds.

4. The smart hearing device for distinguishing natural or non-natural language as described in claim 3, characterized in that the providing unit provides one side of the sound through personalization by setting at least one of the control parameters of an amplification value change due to an environmental change, a volume adjustment, and a frequency adjustment based on the natural or non-natural language identified by the user's hearing data and the matched similar data.

5. 5. The smart hearing device for distinguishing natural or non-natural language according to claim 4, wherein the providing unit sets the control parameters to the sound data of the digital signals received from the first microphone and the second microphone to adjust the balance of at least one of an amplification value change, a volume adjustment, and a frequency adjustment, and converts the digital signal for the adjusted signal into an analog signal to provide one side of the sound to the user.

6. The smart hearing device for distinguishing natural or non-natural language as described in claim 1, characterized in that the feedback transmission unit provides feedback of words and sentences in response to the similar data when the sound data determined by the processing unit to be a natural or non-natural language is matched with the similar data of similar words and sentences.

7. The smart hearing device for distinguishing natural or non-natural language as described in claim 6, characterized in that the feedback transmission unit extracts feedback of words and sentences in response to the user's voice question and transmits it as a voice message when the sound data is matched with the similar data of similar words and sentences based on artificial intelligence.

8. a first smart hearing device that determines whether a first sound data including a voice signal and a noise signal received from a first microphone and a second microphone formed on one side is a natural word or a non-natural word, and provides a sound from one side by setting a first control parameter according to matching of similar data for the determined natural word or non-natural word; a second smart hearing device that determines whether a second sound data including a voice signal and a noise signal received from a third microphone and a fourth microphone formed on the other side is a natural word or a non-natural word, and sets a second control parameter according to matching of the similar data with the determined natural word or non-natural word to provide a sound from the other side; a mobile device for controlling the first smart hearing device and the second smart hearing device; An artificial intelligence hearing system comprising:

9. The artificial intelligence hearing system of claim 8, wherein the first smart hearing device and the second smart hearing device each include the first microphone and the third microphone located near the user's mouth, and the second microphone and the fourth microphone located at a distance from the user's mouth.

10. The artificial intelligence hearing system of claim 8, wherein the first smart hearing device and the second smart hearing device each provide the user with personalized one-side sound and other-side sound, respectively, so that the user can stereoscopically recognize environmental changes and noise changes due to the directionality of the sound from the left and right sides.

11. The artificial intelligence hearing system of claim 8, wherein the first smart hearing device and the second smart hearing device each set the first control parameter and the second control parameter to different parameter values ​​based on the user's left hearing data and right hearing data.

12. The artificial intelligence hearing system of claim 8, wherein the mobile device controls one or more of the power on / off, signal collection, and control parameter settings of each of the first smart hearing device and the second smart hearing device according to a user's selection input.

13. 1. A method of operating a smart hearing device for distinguishing natural or non-natural language, comprising: receiving sound data of a voice signal and a noise signal from a first microphone and a second microphone formed on one side; comparing the digital flow of the sound data with a pre-stored graph pattern to determine whether the sound data is a natural or non-natural language; matching similar data for the determined natural or non-natural language based on a database including a natural language domain and a non-natural language domain; setting a control parameter to the natural or non-natural language identified by the matched similar data, and providing the user with the converted one-side sound; providing feedback in response to the identified natural or non-natural words according to the matched similar data; 1. A method of operating a smart hearing device for distinguishing natural or non-natural language, comprising:

Images