Electronic equipment for identification and direction signaling of a sound signal source for the hearing impaired.

Abstract

Electronic equipment for identifying and signaling the direction of a sound signal source for the hearing impaired. The present invention relates to a microprocessor-based electronic device for determining and alerting the user, primarily the hearing impaired, of the direction (right or left) of origin of a nearby sound source (1) such as a motor vehicle, a horn, a dog barking, a train whistle, etc. The user must wear the device close to their body, so that it will capture ambient sound by means of two (or more) microphones (4, 5), each located on each shoulder of the user, at an average distance of 30 cm from each other. The sound signals captured from the environment by the microphones will have a time lag due to the difference in distance between the sound source and the sensors (microphones).This phase shift will be calculated by software running on a microcontroller (6) which will determine the direction (right or left) of the sound source in relation to the user according to the phase shift obtained. The microcontroller, possessing this information, will activate one of the two (or more) vibracall motors (7, 8) placed on each of the user's shoulders to alert him to the direction from which the sound is coming. The user, instinctively, when feeling a vibration above his shoulder (right or left), will tend to look or turn his neck in the direction determined by the equipment for the sound source and thus become aware of the proximity of the sound source, being able to avoid an accident.

Description

1 / 8 “ELECTRONIC EQUIPMENT FOR IDENTIFICATION AND DIRECTIONAL SIGNALING OF AUDIBLE SIGNAL SOURCE FOR THE HEARING IMPAIRED”

[001] This document refers to the invention of an electronic device that has the function of detecting and alerting the hearing impaired person about the direction of a nearby sound signal source: right or left, especially if it is out of their visual range, i.e., behind the person using it.

[002] Worldwide, 466 million people suffer from hearing loss with disabling consequences. Of this total, 34 million are children. According to the World Health Organization (WHO), this situation could worsen by 2050, when more than 900 million people are expected to experience hearing loss at this same level (WHO INTERNATIONAL NETWORK FOR HEALTH TECHNICIAN EDUCATION warns that hearing loss could affect more than 900 million by 2050. Available at<http: / / www.rets.epsjv.fiocruz.br / noticias / > (Accessed March 2020). In Brazil, according to the 2010 census conducted by the Brazilian Institute of Geography and Statistics (IBGE), 9.7 million people have hearing impairments. Of these, 2.2 million have severe hearing impairment, a situation in which there is a loss of between 70 and 90 decibels (dB) (PORTAL DO TRANSITO). Despite advances, deaf people still face accessibility barriers. Available at:<https: / / portaldotransito.com.br / noticias / > (Accessed in March 2020).

[003] Hearing impairment brings many limitations to the individual's development (REDONDO, MC da F.; CARVALHO, JM. Hearing Impairment. 1st ed. Brasília: MEC, 2000. 64 p. Secretariat of Distance Education. ISSN 1518-4706). Hearing impairment is treated by many authors as the type of disability that is most difficult to manage within the rest of society (PAGLIUCA, LMF; FIÚZA, NLG; REBOUÇAS, CB de A. Aspects of the nurse's communication with the hearing impaired. Revista da Escola de Enfermagem da USP, v. 41, n. 3, p. 411-418, Sept. 2007. Available at:<https: / / www.revistas.usp.br / reeusp / article / view / 41636> (Accessed March 2020). When comparing people with physical, hearing, and visual disabilities, those with hearing impairments face the greatest difficulty in social inclusion, since hearing is the essential sense for acquiring and using language. According to the authors, in light of... Petition 870200050157, dated 04 / 22 / 2020, pages 28 / 38 2 / 8 due to the invisibility of their limitations, this disabled person is often stigmatized as rebellious and deceitful.

[004] The expansion of cities and urbanization, along with the increase in automobiles and mechanical equipment, have made traffic disorganized and subject to various types of unforeseen events that can create problems for people, especially for the disabled. Thus, difficulties arise for people with disabilities in the mobility of public roads and in accessing public transport, since each type of disability requires a certain type of adaptation so that daily tasks can be carried out. Due to this disorganization, one of the challenges faced by the hearing impaired is the perception of sound signals, which can be the sound of an approaching vehicle engine, a horn, a train whistle, among other traffic sounds.

[005] Because the localization of sound sources is an important function for an individual's auditory performance, since in some situations it is the only sense present for the perception of distance, its absence often implies an increased risk of accidents (RODRIGUES, PC, OLIVEIRA, AC, COLAFÊMINA, JF, MENEZES, PL Sound localization in users of individual hearing aids. Rev. CEFAC, vol. 12, n. 13, São Paulo, May / June 2010).

[006] In perceiving the direction of sound, there must be discrimination of very subtle differences in intensity, frequency, and time (OLIVEIRA, AC de et al. Localization of sound sources: the importance of differences in interaural hearing thresholds. Brazilian Society of Speech-Language Pathology, n. 13, p. 7-11, Feb 2008). Generally, spatial sound localization depends on the analysis of differences in these parameters between the sounds that reach the two ears. The localization of sound sources occurs, above all, due to the influence of physical characteristics of the sound wave, which generate differences in the spectrum of this wave between the ears, that is, the so-called interaural differences. Thus, two main factors appear, namely, the phase differences of the sound signal between the ears, called interaural time difference (ITD), and the intensity differences between the ears, called interaural intensity difference (IID).Therefore, it can be stated that people with bilateral hearing loss will hear better if they are using bilateral hearing aids. However, hearing with two behind-the-ear or in-the-ear hearing aids does not always enable or improve sound localization. Petition 870200050157, dated 04 / 22 / 2020, pp. 29 / 38 3 / 8

[007] Aiming to improve the quality of life of people, especially the hearing impaired, this work presents a proposal for the development of an electronic device, consisting of hardware and software, capable of detecting the direction of the source of a sound signal from the phase difference between the signals captured by two or more microphones and alerting the user to possible nearby danger.

[008] The works developed by Luz (LUZ, GBB. Determination of the Phase Shift Between Signals: Comparison and Analysis of Various Practical Methods. Master's Dissertation - Federal University of Rio de Janeiro - Polytechnic School, 2016. Electronic and Computer Engineering Course.) and Andrade (ANDRADE, BC de. Determination of the phase shift between signals using digital processing methods. Master's Dissertation - Federal University of Rio de Janeiro - Polytechnic School, 2017. Electronic and Computer Engineering Course.) exhibit various practical methods for determining the phase shift between two signals, and seek to expose the necessary means for the use of each one, their advantages and disadvantages, in addition to proposing combinations between them and the consequent composition of new methods with the same objective.Furthermore, the works demonstrate how to use the appropriate technique to express analog signals in exponential form before they are processed by digital methods.

[009] The work proposed by (FEREL, Radhian; IKRAM, Fadhli; NOLIKA, Swizya; ADIONO, Trio. Efficient Sound-Source Localization system using low cost TDOA computation. In: International Symposium on Electronics and Smart Devices - ISESD, 2016, p. 315-319. DOI 10.1109 / ISESD.2016.7886741) presents an algorithm for sound source localization, with a Very Large Scale Integration (VLSI) design. The created system supports sound recognition in three-dimensional space, using 4 sensors and the Time Difference Of Arrival (TDOA) computational technique. The system is implemented on an Altera Cyclone II FPGA with Verilog HDL encoding. The authors highlight the system's performance, but there are disadvantages such as the high cost of the FPGA and the high complexity of the system for its implementation.

[010] Patent number CN2103876 deals with a correction method for hearing aids, to allow hearing-impaired individuals who use two hearing aids (sound amplifiers), one in each ear, to better identify the direction of a sound source, since using the same hearing aid in both ears makes it difficult to Petition 870200050157, dated 04 / 22 / 2020, pages 30 / 38 4 / 8 Identification of the direction from which the sound is coming due to the distortion in the phase and amplitude of the sound signal caused by the devices. The equipment is for hearing-impaired individuals who still have a small hearing capacity; that is, it is not suitable for those who are totally deaf. The equipment described in this patent is only for people who use a hearing aid, unlike our proposal which aims to assist totally deaf people.

[011] Patent number CN102387457 deals with equipment that converts sound into vibration, but it does not have the function of identifying the direction of sound as the invention proposed by us does.

[012] Patent number CN203480611 describes a device consisting of 4 identical circuits, arranged at 90 degrees horizontally from the user, fixed to a hat or collar. Each circuit contains a microphone that captures the sound, which is amplified, then filtered by frequency, and finally, the signal is used to activate a vibrating motor when it exceeds a certain level. When one of the 4 motors activates, it alerts the user to the direction of the sound through its vibration. In this patent, since there is no connection between the 4 circuits, more than one motor may vibrate depending on the sound intensity of the source. Thus, it differs from our proposal, as we propose a device with possibly only 2 microphones, and the direction is obtained by calculating the phase difference between the signals captured by the microphones, and not by the signal intensity.In our proposal, the two microphones are positioned on the user's shoulders to generate the necessary signal phase shift for direction detection.

[013] Patent number KR20190037596 concerns a device with 4 microphones arranged on different sides of a garment worn by a hearing-impaired person. The 4 microphones receive sound information from the surroundings, compare the sound characteristics with the sound characteristics stored in the device, and determine the type of sound and its direction, displaying this information on the screen. There is an LED whose purpose is to indicate the degree of danger so that deaf people can see the direction of the sound and its degree of danger. The device also has a gyroscope-type sensor to know the position of the garment and to determine the direction of the sound relative to the gyroscope. The sound direction information obtained through the 4 microphones and the position of the garment obtained through the gyroscope sensor are combined from the user's point of view, and then it is possible to display the direction of the sound on a screen. The direction and type of sound are determined Petition 870200050157, dated 04 / 22 / 2020, pages 31 / 38 5 / 8 of the signal obtained by each microphone. The equipment also has a vibration motor to vibrate depending on the signal detected by the control unit. The gyroscope sensor increases the product's cost, in addition to increasing the system's complexity. In the patent proposed by us, only two microphones and two vibration motors placed on the user's shoulders are needed, indicating the direction of the sound: right or left. In the patent proposed by us, the direction of the sound is determined by calculating the phase shift of the signals captured by the two microphones, which requires a low computational cost. OBJECT OF THE INVENTION

[014] Therefore, the present low-cost and simple-to-operate invention was developed with the aim of minimizing the difficulties encountered by hearing-impaired individuals in determining the direction of origin of a given nearby sound source, such as a vehicle engine, a dog barking, a horn or an object hitting the ground, and thus avoiding a possible accident.

[015] This invention relates to a microprocessor-based electronic device for determining and alerting the user, primarily the hearing impaired, of the direction (right or left) of origin of a nearby sound source. The user must wear the device close to their body, where it will capture ambient sound through two (or more) microphones, each located on each of the user's shoulders at an average distance of 30 cm from each other. The sound signals captured from the environment by the microphones will have a time lag due to the difference in distance between the sound source and the sensors (microphones). This lag will be calculated by software running on a microcontroller, which will determine the direction (right or left) of the sound source in relation to the user, also according to the lag obtained. The microcontroller, possessing this information, will activate one of the two (or more) vibracall motors placed on each of the user's shoulders to alert them of the direction from which the sound is coming.The user, instinctively, when feeling a vibration above their shoulder (right or left), will naturally tend to look or turn their neck in the direction indicated for the sound source and thus become aware of their proximity to the sound source. DESCRIPTION OF THE INVENTION Petition 870200050157, dated 04 / 22 / 2020, pages 32 / 38 6 / 8

[016] The present invention can be better described and understood by means of the attached figures, wherein: FIGURE 1 presents an overview of the system in operation. FIGURE 2 presents the equipment diagram. FIGURE 3 presents the block diagram of the microprocessor-based electronic circuit. FIGURE 4 presents the flowchart of the software executed by the microcontroller. System Overview

[017] Referencing the figures, FIGURE 1 shows an overview of the system's operation, where any sound source (1) generates sound waves (2) from behind an individual (3). These sound waves are captured by two electret microphone sensors (4, 5) placed respectively on the user's right and left shoulders (3). Due to the difference in position between the two microphones, which are approximately 30 cm apart, each sensor will receive the sound at different times due to the finite speed of sound propagation through the air (around 340 m / s). Thus, the sensors will provide two identical signals, but out of phase in time, to the microcontroller circuit (6). The phase difference between the two signals will be proportional to the difference in distance between the sound source and each microphone.In FIGURE 1, the sound source is closer to the left microphone (5), so the signal from the left microphone will be ahead of the signal picked up by the right microphone. The microcontroller (6) receives the amplified and filtered signals from the two microphones, converts them to digital data, and calculates the phase difference between the two signals using an algorithm. Thus, the microcontroller decides which direction the sound is coming from in relation to the user: from the right or from the left. As in the example in FIGURE 1, the sound source is closer to the user's left side, so the microcontroller will decide through its software that the vibracall motor on the user's left side (8) should vibrate (9) to warn them that a sound source is on their left side. The vibracall motor on the right side (7) is inoperative in this example of operation.The user, feeling the vibration from the motor only on their left shoulder, will tend to look or move their neck to that side, thus being alerted to a possible danger and able to take the necessary precautions. Equipment diagram Petition 870200050157, dated 04 / 22 / 2020, pages 33 / 38 7 / 8

[018] FIGURE 2 briefly presents the electronic equipment composed of three separate modules connected by cable. The main module (10) is a circuit board consisting of the microcontroller (11) and accessories, and the power source (12) which can be batteries or rechargeable batteries. The other two secondary modules (13, 14) are identical electronic circuit boards each consisting of an electret microphone (15, 16), signal filter circuit, signal amplifier, current amplifier and vibracall motor (17, 18). The secondary modules (13, 14) are connected to the main module (10) by means of electrical cables (19, 20). Both the main module and the secondary modules are fixed to the user's body by means of a garment (a shirt, or jacket, or vest), where, on each shoulder, there will be a secondary module, and in the central part of the garment will be the main module.These modules and connecting cables can be hidden under clothing as long as the microphones have access to the outside environment. Block diagram of the electronic circuit

[019] FIGURE 3 presents a simplified block diagram of the electronic circuit of the equipment composed of: electret microphones (21, 22) to capture sounds from the user's right and left sides; amplifiers (23, 24) to increase or decrease the sensitivity of the captured signal; low-pass filters (25, 26) to cut unwanted frequencies from the captured signal; microcontroller (27) responsible for converting analog signals into digital signals and for executing the software that calculates the phase shift between the two signals and decides the direction of the captured sound (right or left); current amplifier (28, 29) that amplifies the microcontroller's command; and the vibracall motors (30, 31) that are activated, one at a time, depending on the result obtained by the microcontroller. The power supply and other components have been omitted from the figure for simplification. Software flowchart

[020] According to the system overview and the block diagrams shown, the implemented algorithm, as shown in the flowchart in FIGURE 4, has the function of processing the sound signals captured by the microphones and acquired by the microcontroller. The algorithm starts (32) by initializing the program variables (33) and Petition 870200050157, dated 04 / 22 / 2020, pages 34 / 38 8 / 8 configuring the microcontroller's internal analog-to-digital converter (34), ADC. Then, readings (35) are taken from the two analog channels of the microcontroller where the outputs of the two microphones (right and left) are connected. The values ​​acquired by the two channels are used to calculate the energy (36) of the captured sound. If the resulting energy is less (37) than a specified value, the algorithm performs the acquisitions from the two channels again (35). If the resulting energy is greater (37) than a specified value, the algorithm acquires a predefined number of samples from each channel (38) to perform the phase shift calculation (39). This calculation can be done using some computational intelligence technique (such as Artificial Neural Networks), or by calculating the correlation between the signals using vector product, or even by some other technique implemented in software.Based on the obtained phase difference, the software decides which vibracall motor should be activated (40): the right or the left to indicate to the user the direction of origin of the sound. The selected vibracall motor is activated for a short period of time, just long enough for the user to notice and take some action. Before the program repeats its execution, a time interval of a few seconds is expected (41). Petition 870200050157, dated 04 / 22 / 2020, pages 35 / 38

Claims

1 / 1 CLAIMS 1. Electronic equipment for identifying and signaling the direction of a sound signal source for the hearing impaired, comprising an embedded hardware and software system that includes a main module (6, 10) with a microcontroller (11, 27), auxiliary circuits and battery (12), as well as two secondary modules (13, 14) each containing a microphone (15, 16) with amplifier (23, 24) and filter (25, 26) and a vibracall type vibratory motor (17, 18) with current amplifier (28, 29), characterized by comprising embedded software for the purpose of determining the direction of the origin of a sound signal by means of the software and indicating the result to the user; the user uses it with a garment over his body (3), in such a way that the two microphones (4, 5) and the two vibracall motors (7, 8) are each on his shoulders, and the rest of the equipment (6, 10) is hidden inside the garment; and determine the direction of origin (1) of the sound signal (2), based on the calculation of the time phase shift of the signals picked up by the two microphones (4, 5) installed each on the user's right and left shoulders (3), whose distance between the microphones causes the phase shift necessary to allow its calculation. Petition 870260065029, dated 02 / 07 / 2026, page 4 / 4

Images