Sound somatosensory wearable device

Abstract

The invention belongs to the field of wearable intelligent equipment, and particularly relates to a sound somatosensory wearable device. The device comprises an annular body, a sound receiver, a vibrator and a processor, wherein the sound receiver, the vibrator and the processor are mounted on the annular body; the sound receiver is used for receiving sound signals in all directions; the processor calculates the frequency, the intensity and the direction of the sound signals according to the sound signals in all the directions, and calculates the intensity, the position and the frequency of the vibration signals according to the frequency, the intensity and the direction of the sound signals; the vibrator is used for vibrating according to the position, strength and frequency of the vibration signal. The direction, strength and frequency of sound are obtained from multiple pieces of sound information through an algorithm through the sound receiver, and the sound information is converted into multi-channel vibration signals, so that the vibration signals can be transmitted to the skin through the wearable device to be perceived by touch, the sound perception ability and experience of people can be supplemented or enhanced, and the immersive reality sense can also be enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of wearable intelligent devices, and in particular relates to a sound somatosensory wearable device. Background Art

[0002] At present, there are more than 28 million hearing-impaired people in my country, accounting for about 2% of the national population. 5% of the world's population, more than 400 million people suffer from moderate or above hearing impairment. It is the largest type of disability among all types of disabled groups, so solving the difficulties of hearing-impaired people is a major social need.

[0003] Currently, hearing aids and cochlear implants are the two main ways to restore hearing to the hearing-impaired: Hearing aid: Hearing aids are mainly composed of a shell, microphone, amplifier, receiver and power supply. Its main working principle is: wear it on the ear through the shell; collect sound through the microphone and convert it into radio waves; increase the intensity of the radio waves through the amplifier; play the amplified sound through the receiver; and feel the sound through the user's residual hearing.

[0004] Cochlear implant: The basic structure includes an external part and an implant part. The external part includes a microphone, a speech processor, a transmitting coil and connecting wires; the implant part includes a receiving coil, a stimulator and electrodes. The electrode system implanted in the body directly stimulates the auditory nerve to restore or reconstruct the hearing function of the deaf. It is often used clinically to treat patients with severe or profound sensorineural hearing loss. Its main working principle is: collect external sounds through a microphone; convert the sound into a coded form of electrical signal; transmit it to the auditory nerve to stimulate the nerve, and then transmit it to the brain to produce hearing.

[0005] Hearing impairment is generally divided into four levels: mild, moderate, severe, and extremely severe. For people with mild and moderate hearing impairment who have residual hearing, hearing aids can usually restore hearing. However, for severe hearing impairment, high-end but expensive hearing aids are required. For extremely severe hearing impairment, that is, people who are completely deaf, hearing aids will not work. This is because the principle of hearing aids is to collect sound through a microphone and then amplify the sound through an amplifier and transmit it to the ear using residual hearing. Therefore, hearing aids are not available for people without residual hearing.

[0006] For sensorineural hearing impairment, it is generally treated by surgical implantation of a cochlear implant. The current cost of surgery plus cochlear implant ranges from RMB 100,000 to RMB 300,000. Moreover, it cannot be installed for people with symptoms such as ruptured eardrum and ear canal deformity. Because the cochlear implant is an electronic device, it also needs to be charged, replaced, and maintained. Finally, the implant surgery is usually accompanied by follow-up treatment and a certain risk of infection.

[0007] To summarize, hearing aids and cochlear implants are the two main solutions to hearing impairment. However, due to differences in indications, prices, portability, safety, etc., their acceptance and popularity among the hearing-impaired population are low. Summary of the invention

[0008] In order to solve the above-mentioned problems of hearing aids and cochlear implants, the present invention proposes a wearable sound sensory device by utilizing the principle of converting sound into vibrations to allow users to feel the sound through their sense of touch.

[0009] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions: A sound body sensing wearable device comprises a ring-shaped body, a sound receiver, a vibrator and a processor mounted on the ring-shaped body, The microphone is used to receive sound signals from all directions; The processor calculates the frequency, intensity, and direction of the sound signal according to the sound signals in various directions, and calculates the intensity, position, and frequency of the vibration signal according to the frequency, intensity, and direction of the sound signal; The vibrator is used to vibrate according to the position, intensity and frequency of the vibration signal.

[0010] Preferably, the sound receiver includes a plurality of sound receiving units, and the sound receiving units are distributed in an array.

[0011] Preferably, the vibrator comprises a plurality of vibration units, and the vibration units are distributed in an array.

[0012] Preferably, the sound receiving units are evenly distributed on one side of the annular body, and the vibration units are evenly distributed on the other side of the annular body opposite to the sound receiving units.

[0013] Preferably, the processor determines a sound waveform for generating vibration according to the frequency of the sound signal; converts the intensity of the sound signal into the intensity of the vibration signal according to the sound waveform; and converts the direction of the sound signal into the position of the vibration signal.

[0014] Preferably, the method for obtaining the intensity of the vibration signal comprises the following steps: The sound analog signal waveforms of multiple receiving channels are collected synchronously; the sound analog signal waveform of each receiving channel is subjected to FFT transformation, and the frequency characteristics of each receiving channel are obtained through FFT transformation calculation; the frequency with the highest amplitude in the receiving channel is found as the typical frequency, and the waveform corresponding to the typical frequency is the typical waveform of the receiving channel, and the strength of the receiving channel is calculated according to the amplitude of the typical waveform.

[0015] Preferably, the method for obtaining the direction of the sound signal comprises the following steps: Each receiving channel extracts the sound analog signal waveform corresponding to the sound of the same frequency, and then calculates the phase difference of the sound analog signal waveform collected by each receiving channel through FFT transformation. The time difference for each receiving channel to obtain the same sound analog signal waveform is obtained through the phase difference. The distance from each receiver to the sound source is obtained by multiplying the time difference by the speed of sound. When the position of each receiver is known, an overdetermined equation is listed according to the position of each receiver and the distance from each receiver to the sound source, and the position coordinates of the sound source are calculated to determine the direction of the sound.

[0016] Preferably, the processor is also used to recognize the voice of words and / or sentences based on the sound signal, and convert the voice of the words and / or sentences into a vibration combination.

[0017] Preferably, it also includes a wireless module, which is used to realize communication between the mobile terminal and the processor, and is used to output control instructions or receive data.

[0018] Preferably, the sound receiver includes a vibrator, a motor and an electrode sheet.

[0019] Compared with the prior art, the present invention has the following beneficial effects: the device of the present invention obtains the direction, strength, and frequency of sound through a microphone, and converts it into a multi-channel vibration signal, so that the vibration signal can be transmitted to the skin through a wearable device for tactile perception, which can supplement or enhance people's perception and experience of sound, and can also enhance the sense of reality of being in the scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of the structure of a sound somatosensory wearable device in Example 1; Figure 2 This is a principle block diagram of the sound body sensing wearable device of Example 1; Figure 3 is a circuit diagram of the microphone array in Example 1; Figure 4 is a flow chart of the method for sound position recognition in Example 2; Figure 5 is a schematic diagram of the microphone array in Example 3; Figure 6 is a rear oblique viewing angle diagram of the microphone array in Example 3; Figure 7 is a diagram of the vibrator array in Example 3; Figure 8 This is a rear oblique view of the vibrator array in Example 3.

[0021] Figure numerals: 1-sound sensory collar; 2-first microphone; 3-second microphone; 301-rearward ambient sound microphone; 302-right ambient sound microphone; 303-left ambient sound microphone; 4-control button; 5-first vibration component; 6-second vibration component; 7-third vibration component; 8-fourth vibration component; 9-fifth vibration component. DETAILED DESCRIPTION

[0022] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0023] Example 1 A sound sensor wearable device such as Figure 1 As shown, it includes an annular body (sound sensor collar 1), a receiver (a first microphone 2 and a second microphone 3) mounted on the annular body, a vibrator (numbered 5-9) and a processor (the processor is embedded in the annular body 1, Figure 1 The receiver is used to receive sound signals from all directions; the processor calculates the direction, intensity, and frequency of the sound signals according to the sound signals from all directions, and calculates the position, intensity, and frequency of the vibration signal according to the direction, intensity, and frequency of the sound signals; the vibrator is used to vibrate according to the position, intensity, and frequency of the vibration signal. A control button 4 is also included for manual control.

[0024] Based on the above principle, a sound somatosensory collar is provided, which includes an elastic collar body, a microphone array with specific distribution on the outside of the collar body, a vibrator array with uniform distribution on the inside of the collar body, a processor chip and a battery embedded in the collar body, and an operation button set on the collar body. The working principles of its main components are as follows: The circular microphone array collects sound signals from all directions of the body (including the user's own voice) and transmits them to the processor chip; the processor chip receives sound signals from multiple directions, and then combines the sound signals with the position and direction parameters of each microphone and the time difference of the sound signals to calculate the direction, intensity and frequency of the sound signals.

[0025] The processor chip calculates the position, strength and frequency of the vibration signal based on the direction, strength and frequency of the sound signal. The position of the vibration signal corresponds to the number of the microphone that needs to vibrate, the strength of the vibration signal corresponds to the vibration strength of the vibrator, and the frequency of the vibration signal corresponds to the vibration frequency.

[0026] Preferably, the device further comprises a wireless module, which is embedded in the collar, and the wireless module is used to realize communication between the mobile terminal and the processor, and is used to output control instructions or receive data. Specifically, the mobile terminal can be a mobile phone, a tablet computer or a phone watch. Taking a mobile phone as an example, the application in the mobile phone is connected to the collar through the wireless module. When the function corresponding to the mobile phone application is clicked, the preset vibration data is sent to the processor chip of the collar through the wireless module.

[0027] The processor chip loops through the steps of receiving the sound signal, calculating and outputting the vibration signal, and outputs the vibration signal of the corresponding direction in real time according to the source direction of the sound signal. Figure 2 shown.

[0028] The circuit diagram of the microphone array is as follows: Figure 3 shown.

[0029] Depending on the situation, the processing flow of converting sound signals into vibration signals is different, as follows: The first case, natural environmental sound perception Various sounds emitted in the natural environment are combined with human voices into original sound waves; the microphone array receives multiple original sound waves, generates multiple electrical signals and sends them to the processor chip; the processor chip uses an amplifier to amplify and process the multiple electrical signals to ensure signal strength and stability, and converts the electrical signals into digital signals.

[0030] Because each microphone is located in a different position and facing a different direction, even for the same sound source, each sound signal arrives at the microphone at a different time, and the intensity of each sound signal arriving at the microphone in the microphone array is also different. The processor chip calculates the direction, intensity, and frequency of the sound by taking into account the time difference, intensity difference, and phase difference of multiple sound signals and the known microphone positions.

[0031] After obtaining the direction, intensity and frequency of the sound, the processor chip first calculates the number of the vibrator at that direction through the azimuth angle, converts the intensity into the vibration intensity supported by the vibrator, and converts the sound frequency into the vibration frequency supported by the vibrator.

[0032] The output of vibration intensity and vibration frequency is generated by the processor sending the converted digital signal to the driver chip of the corresponding vibrator, and the driver chip sends a vibration instruction to the vibrator.

[0033] The second case, human voice perception Various sounds emitted from the natural environment are combined with human voices to form original sound waves; the microphone array receives multiple original sound waves, generates electrical signals and sends them to the processor; the processor uses an amplifier to amplify and process the electrical signals to ensure signal strength and stability and convert them into digital signals; The processor extracts the words and sentences of people speaking from the audio signal collected by the microphone through noise reduction and recognition. The processor chip recognizes the voice of words and sentences and converts them into vibration combinations word by word. The digital signals of the vibration combinations are then sent to the vibrator array one by one to generate vibration combinations of corresponding intensities. By learning the vibration combinations, users can use this method to know the content of the other person's speech during the conversation. For example, when someone in a certain direction says "thank you", the processor chip recognizes the voice of "thank you" and converts it into a vibration signal. The frequency of the vibration signal is 5 times / s, and the intensity is the first level intensity (the first level intensity is the weakest intensity). Through advance learning, the person wearing the collar knows that someone said "thank you" to him in the direction of the vibrator.

[0034] The third case: vocal feedback Because hearing-impaired people have lost their hearing, they cannot receive real-time feedback to correct their voice when they speak.

[0035] The user makes a sound with his throat, which becomes a raw sound wave. The microphone array receives the raw sound wave. The processor extracts the words and sentences spoken by the person through noise reduction and recognition of the audio signal collected by the microphone. The processor chip recognizes the voice of the words and sentences and converts them into vibration combinations word by word. The digital signals of the vibration combinations are then sent to the vibrator array one by one to generate vibration combinations of corresponding intensities. The user can sense the changes in the vibration intensity and combination when he speaks, and can correct his pronunciation.

[0036] Sound is an important part of people's reception of external information. Various sounds in the environment and conversations contain information such as the direction, distance, strength, and frequency of the sound source. People receive sounds through their ears and interpret various meanings through their brains, thus completing their behaviors such as perceiving the environment and communicating with others.

[0037] Therefore, by using a microphone array to obtain multiple sound information through an algorithm, such as the direction, distance, strength, frequency, etc. of the sound, and converting them into multi-channel vibrations or electrical signals according to certain rules, and transmitting them to the skin through wearable devices for tactile perception, it can supplement or enhance people's perception and experience of sound, and also enhance the sense of immersion.

[0038] Through this device, hearing-impaired people will be able to recognize cars, reminders and other basic environmental sound perception capabilities by wearing a sound sensory collar. With the help of software or the guidance of a trainer, they can have the ability to recognize words, short sentences and even long sentences.

[0039] Currently, hearing-impaired people can only feel the vibration and learn to speak by touching their throats with their hands. Through this device, hearing-impaired people can obtain feedback on the size and frequency of their own voices through the vibration of the collar during the vocalization process to correct the correctness of their own voices.

[0040] Through the above means, the hearing-impaired can be assisted in sound perception and pronunciation, thereby improving the effectiveness and efficiency of communication between the hearing-impaired and normal-hearing people.

[0041] Example 2 In the process of converting sound into vibration signal, the flow chart of the method of sound direction recognition is as follows Figure 4 As shown. For each sound receiver in the microphone array, after receiving the sound signal, the sound signal is output to the ADC converter to realize the conversion from analog signal to digital signal, obtain the sound digital signal, then convert the digital signal into analog signal, and filter the analog signal. Through algorithm processing, the frequency, amplitude and phase of the sound signal received by each sound receiver are calculated, so as to obtain the phase difference of the multi-channel sound signal. Then the distance from the sound source to each microphone is calculated, the position of the microphone in the virtual coordinate system is calculated, and the information output by each vibrator is calculated according to the position of the sound source and the sound amplitude received by the microphone. According to the information output by each vibrator, the driver sends a vibration instruction to the vibrator to generate vibration.

[0042] Specifically, the method for calculating the vibration intensity includes: the processor synchronously collects the sound data (i.e., the sound analog signal waveform) of multiple microphone channels in the microphone array, and then performs FFT (Fourier transform) on the sound analog signal waveform of each channel, and obtains the frequency characteristics of each channel through FFT calculation (there may be multiple frequency characteristics). The frequency characteristic with the highest amplitude in the channel signal is found as the typical frequency, and the waveform corresponding to this frequency is the typical waveform of the channel. The intensity of the channel can be calculated by the amplitude of the typical waveform. The amplitude of the typical waveform collected by each channel is different, and this amplitude is the embodiment of the vibration intensity of the sound intensity in each vibrator.

[0043] Furthermore, the direction of the sound is determined by calculating the time difference of receiving the same sound at different microphones, including: each channel extracts a typical waveform of the same frequency sound, and then calculates the phase difference of the typical waveform collected by each channel through the FFT value (the phase difference is the time difference from the sound to each microphone), and the distance from each microphone to the sound is obtained by multiplying the time difference by the speed of sound. If the position of each microphone and the distance from each microphone to the sound are known, an overdetermined equation can be listed to calculate the position coordinates of the sound source, thereby determining the direction of the sound. After calculating the distance from each microphone to the sound, calculating the position of the sound source includes the following steps: determining a spatial coordinate system in the software, all relative position relationships are based on the spatial coordinate system, and the positions of each microphone and vibrator in this spatial coordinate system are known and fixed, and then the distance from the sound to each sensor, the position coordinates of the microphone and the vibrator are taken as known quantities, and the three-dimensional coordinates (x, y, z) of the sound position are taken as unknown quantities, and the equation is listed for calculation, so that the three-dimensional coordinates of the sound in this spatial coordinate system can be obtained, thereby determining the direction of the sound.

[0044] After obtaining the frequency, intensity and direction of the sound, the vibration of the vibrator can be controlled according to the corresponding output rules. For example, the output rule is: the closer to the sound source, the higher the vibration intensity and frequency of the vibrator output. In this way, for the same sound source, a vibration point with the highest vibration intensity and frequency is formed in the entire vibration array as the center, and a number of vibration points form a vibration surface. This vibration surface stimulates the skin surface of the subject's neck, etc., forming a sense of touch, thereby converting sound into vibration and transmitting information to the subject.

[0045] Example 3 A sound sensor collar, the structure of which is shown in the figure Figure 5-8 As shown. There are several grooves on the outside of the sound somatosensory collar, and a microphone is embedded in each groove for collecting sound. Preferably, the microphones are distributed on the outside of the annular collar, and each microphone faces a direction, for example, respectively distributed in the front, back, left and right with the collar as the center. A forward ambient sound microphone (also known as the second microphone) 3, a rearward ambient sound microphone 301, a right ambient sound microphone 302, and a left ambient sound microphone 303 are formed. Preferably, the first microphone 1 is used as a dedicated microphone for human voice recognition, which is used to recognize the sounds and sentences produced by human speech.

[0046] The microphone array is obliquely viewed from behind. Figure 6 As shown, from Figure 6It can be seen that five microphones form a microphone array, which are distributed on the outside of the sound sensing collar 1. The microphone, as a sound receiving device, is generally cylindrical in shape, but is not limited to a cylindrical shape. The cross-section of the microphone is not limited to a circle or a rectangle. Components in the shape of sheets, dots, etc. that can be installed on the sound sensing collar 1 for sound reception are included in the conception of the present invention and belong to the protection scope of the present invention.

[0047] The schematic diagram of the vibrator array is shown in Figure 7 As shown, the vibrator array is distributed inside the sound sensor collar 1 and contacts the skin. The tactile components of the vibrator array for generating vibrations include but are not limited to vibrators, motors, and electrodes. Figure 8 As shown, the shape of the tactile component is not limited to a rectangle or a circle, and any component that can be installed on the sound sensor collar 1 to generate a vibration signal is included in the concept of the present invention and belongs to the protection scope of the present invention.

[0048] The specific implementation examples described in this article are merely examples of the spirit of the present invention. Those skilled in the art of the present invention may make various modifications or supplements to the specific implementation examples described or take corresponding measures, but they will not deviate from the spirit of the present invention or exceed the scope defined by the attached claims.

Claims

1. A sound sensor wearable device, characterized in that: It comprises an annular body, a receiver, a vibrator and a processor mounted on the annular body, The microphone is used to receive sound signals from all directions; The processor calculates the frequency, intensity, and direction of the sound signal according to the sound signals in various directions, and calculates the intensity, position, and frequency of the vibration signal according to the frequency, intensity, and direction of the sound signal; The vibrator is used to vibrate according to the position, intensity and frequency of the vibration signal.

2. A sound somatosensory wearable device as claimed in claim 1, characterized in that: The sound receiver includes a plurality of sound receiving units, and the sound receiving units are distributed in an array.

3. A sound somatosensory wearable device as claimed in claim 2, characterized in that: The vibrator comprises a plurality of vibration units, and the vibration units are distributed in an array.

4. A sound somatosensory wearable device as claimed in claim 3, characterized in that: The sound receiving units are evenly distributed on one side of the annular body, and the vibration units are evenly distributed on the other side of the annular body opposite to the sound receiving units.

5. The sound sensor wearable device according to claim 1, characterized in that: The processor determines a sound waveform for generating vibration according to the frequency of the sound signal; converts the intensity of the sound signal into the intensity of the vibration signal according to the sound waveform; and converts the direction of the sound signal into the position of the vibration signal.

6. A sound somatosensory wearable device as claimed in claim 5, characterized in that: The method for obtaining the intensity of the vibration signal comprises the following steps: The sound analog signal waveforms of multiple receiving channels are collected synchronously; the sound analog signal waveform of each receiving channel is subjected to FFT transformation, and the frequency characteristics of each receiving channel are obtained through FFT transformation calculation; the frequency with the highest amplitude in the receiving channel is found as the typical frequency, and the waveform corresponding to the typical frequency is the typical waveform of the receiving channel, and the strength of the receiving channel is calculated according to the amplitude of the typical waveform.

7. The sound sensor wearable device according to claim 5, characterized in that: The method for obtaining the direction of the sound signal comprises the following steps: Each receiving channel extracts the sound analog signal waveform corresponding to the sound of the same frequency, and then calculates the phase difference of the sound analog signal waveform collected by each receiving channel through FFT transformation. The time difference for each receiving channel to obtain the same sound analog signal waveform is obtained through the phase difference. The distance from each receiver to the sound source is obtained by multiplying the time difference by the speed of sound. When the position of each receiver is known, an overdetermined equation is listed according to the position of each receiver and the distance from each receiver to the sound source, and the position coordinates of the sound source are calculated to determine the direction of the sound.

8. A sound somatosensory wearable device as described in any one of claims 1 to 7, characterized in that: The processor is also used to recognize the voice of words and / or sentences according to the sound signal, and convert the voice of words and / or sentences into vibration combinations.

9. The sound sensor wearable device according to claim 7, characterized in that: It also includes a wireless module, which is used to realize communication between the mobile terminal and the processor, and is used to output control instructions or receive data.

10. The sound somatosensory wearable device according to claim 1, characterized in that: The sound receiver includes a vibrator, a motor and an electrode sheet.

Images