Sound tactile feedback system and working method thereof

Abstract

The embodiment of the invention discloses a sound tactile feedback system and a working method thereof, the system comprises a transmitting assembly and a receiving assembly, the transmitting assembly comprises a microphone and a transmitting processing chip, and the microphone is used for collecting sound signals in an external environment; the emission processing chip is used for carrying out sound type classification on the sound signals acquired by the microphone and coding the sound signals into corresponding data packets; the receiving assembly comprises a shell, a receiving chip and a vibration unit, the receiving chip and the vibration unit are arranged in the shell, an opening is formed in one end of the shell, the vibration unit is located at the opening, and silica gel covers the opening in a sealing mode; the receiving chip is used for receiving the data packet and generating a corresponding vibration signal according to the data packet, and the vibration signal comprises a vibration mode and vibration intensity; and the vibration unit is used for working according to the vibration signal so as to generate tactile feedback. The system provided by the embodiment of the invention is convenient to carry and use, and meets special requirements of different users.

Description

Technical Field

[0001] The present invention relates to the technical field of auxiliary equipment for hearing-impaired persons, and in particular to a sound tactile feedback system and a working method thereof. Background Art

[0002] People with hearing impairments need the help of external devices when listening to music. Existing auxiliary devices include wearable music sensing devices, which mainly use vibrations to convey musical characteristics to help hearing impaired people perceive sound through touch. Such devices are usually composed of a vibration unit, a signal receiver, a power supply and a controller. The vibration unit is usually installed on the back of the device. They analyze audio signals, especially the low-frequency part, convert these signals into vibration patterns, and then transmit sound information through the user's skin, allowing the user to feel the rhythm and rhythm of the music.

[0003] However, the vibration modules used in existing devices are usually large in size. Although the vibration transmission effect is taken into consideration, the importance of appearance and wearing comfort is often ignored, and they are not suitable for various occasions. Moreover, different users have different perceptions and preferences for vibration. The device can only receive vibration in a fixed position, and the vibration position cannot be adjusted to meet the special requirements of each user.

[0004] Therefore, it is necessary to design a new system that is easy to carry and use, and can be placed in a user-specified location for use according to the user's personalized needs to meet the special needs of different users. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and provide an audio tactile feedback system and a working method thereof.

[0006] To solve the above technical problems, the objective of the present invention is achieved through the following technical solutions: providing a sound tactile feedback system, comprising: a transmitting component and a receiving component, the transmitting component comprising a microphone and a transmitting processing chip, the microphone being used to collect sound signals in the external environment; the transmitting processing chip being used to classify the sound signals collected by the microphone into sound types, and encode them into corresponding data packets, and send them to the receiving component; the receiving component comprising a shell, a receiving chip and a vibration unit, the receiving chip and the vibration unit being built in the shell, one end of the shell being provided with an opening, the vibration unit being located at the opening, and the opening being covered with silicone; the receiving chip being used to receive the data packet and generate a corresponding vibration signal according to the data packet, wherein the vibration signal comprises a vibration pattern and a vibration intensity; the vibration unit being used to work according to the vibration signal to generate tactile feedback.

[0007] Its further technical solution is: the model of the microphone is MAX9814.

[0008] A further technical solution is: the transmitting processing chip sends the data packet to the receiving chip through the ESP-NOW protocol.

[0009] A further technical solution is: the transmitting processing chip is used to classify the sound type of the sound signal collected by the microphone according to the set signal threshold, encode it into a corresponding data packet, and send it to the receiving component.

[0010] Its further technical solution is: the model of the receiving chip is ESP8266MINI.

[0011] A further technical solution is: the receiving chip is connected to the vibration unit through a current control module.

[0012] A further technical solution is: the receiving component also includes a battery, and the battery is connected to the receiving chip and the vibration unit.

[0013] A further technical solution is that the outer edge of the silicone extends away from the center point of the silicone to form an extension portion.

[0014] In addition, in order to overcome the defects of the prior art, the present invention also provides a working method performed by the above-mentioned sound tactile feedback system, which is characterized by comprising:

[0015] The user sticks the receiving component anywhere on the body, and the microphone collects sound signals from the external environment;

[0016] The transmitting processing chip classifies the sound signal collected by the microphone into sound types, encodes them into corresponding data packets, and sends them to the receiving component;

[0017] The receiving chip receives the data packet and generates a corresponding vibration signal according to the data packet, wherein the vibration signal includes a vibration mode and a vibration intensity;

[0018] The vibration unit operates according to the vibration signal to generate tactile feedback.

[0019] A further technical solution is that the transmitting processing chip classifies the sound signal collected by the microphone into sound types, encodes it into corresponding data packets, and sends it to the receiving component, including:

[0020] The transmitting processing chip classifies the sound signal collected by the microphone according to the set signal threshold, encodes it into corresponding data packets, and sends it to the receiving component.

[0021] The beneficial effects of the present invention compared with the prior art are as follows: the present invention sets a transmitting component and a receiving component, wherein the transmitting component includes a microphone for collecting sound signals in the external environment, and the transmitting processing chip classifies and encodes the sound signals collected by the microphone to form data packets, and sends these data packets to the receiving component; the receiving component includes a shell 21 for accommodating a receiving chip and a vibration unit, one end of the shell is provided with an opening and is covered with silicone, the receiving chip is used to receive the data packet sent by the transmitting component and generate a corresponding vibration signal, the vibration signal includes a vibration mode and a vibration intensity; the vibration unit is used to work according to the received vibration signal, generate tactile feedback, and provide a corresponding vibration experience, so as to achieve easy carrying and use, and can be placed at a user-specified location for use according to the user's personalized needs, so as to meet the special needs of different users.

[0022] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0024] Figure 1 A schematic block diagram of a sound tactile feedback system provided by an embodiment of the present invention;

[0025] Figure 2 A schematic structural diagram of a sound tactile feedback system provided by an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a use state of a sound tactile feedback system provided by an embodiment of the present invention;

[0027] Figure 4 A schematic flow chart of a working method of a sound tactile feedback system provided by an embodiment of the present invention;

[0028] Description of the symbols in the figure:

[0029] 10. Transmitter component; 11. Microphone; 12. Transmitter processing chip; 20. Receiving component; 21. Shell; 22. Receiving chip; 23. Vibration unit; 24. Battery; 25. Silicone. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0032] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0033] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0034] The vibration modules used in the tactile feedback systems in the prior art are usually large in size. Although the vibration transmission effect is taken into consideration, the importance of appearance and wearing comfort is often ignored, and they are not suitable for various occasions. Moreover, different users have different perceptions and preferences for vibrations. The device can only receive vibrations in a fixed position, and the vibration position cannot be adjusted to meet the special requirements of each user.

[0035] In order to solve the problem that the existing equipment cannot fully convey the details of music in terms of vibration intensity and accuracy, the tactile feedback system of this embodiment will be improved and optimized from the following aspects: 1. The introduction of multi-band vibration units 23: The vibration unit 23 is divided into multiple frequency bands, and each vibration unit 23 is responsible for the conversion of music signals in a specific frequency range. By decomposing and processing music in different frequency bands, it is ensured that the system can provide more accurate vibration feedback for the music details in the low-frequency, medium-frequency and high-frequency bands, thereby improving the accuracy and expressiveness of the vibration intensity. 2. Dynamic vibration intensity adjustment algorithm: An intelligent algorithm is used to adjust the vibration intensity in real time according to the dynamic changes of the music. For example, in the part with strong bass, the vibration unit 23 will increase the intensity, while in the delicate high-pitched part, the vibration will be weakened accordingly to achieve accurate transmission of music details. 3. Multi-point tactile feedback design: Considering the different sensitivity of different users to vibration, the system supports multiple vibration units 23 acting on different parts of the body at the same time. By using vibration units 23 in different positions, the rhythm and level changes in complex music fragments can be more comprehensively transmitted, avoiding the vibration of a single position from covering all the details of the music. 4. Vibration feedback synchronization optimization: Through the synchronization optimization of vibration feedback, the vibration feedback can be highly consistent with the audio signal when the key rhythm or strength of the music changes. This precise synchronization will further enhance the user experience in complex music clips.

[0036] To this end, this embodiment concentrates the vibration unit 23 in the shell 21 and uses it by gluing silicone 25, which takes into account both the vibration transmission effect and the importance of appearance and wearing comfort. It is suitable for different positions of users and can be pasted at the required position for use according to user needs, adapting to the special requirements of each user.

[0037] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0038] See also Figure 1 , Figure 1 A schematic block diagram of a sound tactile feedback system provided by an embodiment of the present invention. The sound tactile feedback system can be used in various occasions, including outdoor and indoor occasions, including concert halls, etc.

[0039] See also Figure 1 and Figure 2The sound tactile feedback system includes: a transmitting component 10 and a receiving component 20. The transmitting component 10 includes a microphone 11 and a transmitting processing chip 12. The microphone 11 is used to collect sound signals in the external environment; the transmitting processing chip 12 is used to classify the sound signals collected by the microphone 11 into sound types, encode them into corresponding data packets, and send them to the receiving component 20; the receiving component 20 includes a shell 21, a receiving chip 22 and a vibration unit 23. The receiving chip 22 and the vibration unit 23 are built in the shell 21. An opening is provided at one end of the shell 21. The vibration unit 23 is located at the opening, and the opening is covered with silicone 25; the receiving chip 22 is used to receive the data packet and generate a corresponding vibration signal according to the data packet, wherein the vibration signal includes a vibration mode and a vibration intensity; the vibration unit 23 is used to work according to the vibration signal to generate tactile feedback.

[0040] In this embodiment, the microphone 11 collects sound signals in the external environment, the transmitting processing chip 12 classifies and encodes these sound signals, and then sends the processed data packets to the receiving component 20. The receiving component 20 is composed of a shell 21, a receiving chip 22 and a vibration unit 23. The shell 21 has a built-in receiving chip 22 and a vibration unit 23. There is an opening at one end of the shell 21. The vibration unit 23 is located at the opening and is sealed with a silicone 25. The receiving chip 22 receives the data packet and generates a corresponding vibration signal, which includes a vibration mode and a vibration intensity. The vibration unit 23 vibrates according to the vibration signal, thereby generating tactile feedback. It can convert sound information into tactile feedback, so that hearing-impaired people or users with special needs can perceive environmental sounds or music through touch. This tactile feedback can help users better perceive the existence and characteristics of sound, and improve their perception of sound and quality of life. At the same time, the personalized settings of the system can adjust the vibration mode and intensity according to the needs of the user to provide a customized experience.

[0041] The silicone 25 can be attached to any position of the user, and the entire system is small in size, making it easy to carry and use.

[0042] In one embodiment, see Figure 2 The model of the microphone 11 is MAX9814.

[0043] In one embodiment, the transmitting processing chip 12 sends the data packet to the receiving chip 22 via the ESP-NOW protocol.

[0044] In one embodiment, the transmitting processing chip 12 is used to classify the sound type of the sound signal collected by the microphone 11 according to a set signal threshold, encode it into a corresponding data packet, and send it to the receiving component 20.

[0045] The MAX9814 microphone 11 can capture sound signals in the environment. Its built-in automatic gain control (AGC) function enables it to adapt to sounds of different intensities, including background noise, voice, and music. The AGC function can automatically adjust the gain of the microphone 11 to ensure the stability and clarity of the sound signal. No matter how the intensity of the sound changes, the captured sound signal is originally an analog signal, that is, a continuous voltage change, representing the strength and waveform of the sound.

[0046] Next, the captured analog signal undergoes preliminary amplification processing to increase the signal strength so that subsequent processing is more accurate; specifically, the amplified analog signal is converted into a digital signal by an analog-to-digital converter (ADC) of the transmission processing chip 12. The function of the ADC is to convert a continuous analog signal into a discrete digital signal for easy processing and analysis by a computer.

[0047] After conversion to a digital signal, the transmitting processing chip 12 will classify the sound type by the set threshold. For example, the threshold can be set to thresholdHigh=550 and thresholdLow=250. These thresholds are used to distinguish different types of sound signals: above the threshold (thresholdHigh): if the value of the digital signal is higher than the high threshold, the system may recognize it as music or a stronger sound signal; below the threshold (thresholdLow): if the value of the digital signal is lower than the low threshold, the system may recognize it as speech or a weaker sound signal.

[0048] By adjusting the threshold, you can optimize the system's recognition accuracy for different sound categories. For example, setting the threshold higher or lower can improve the system's ability to distinguish between background noise and specific sound types.

[0049] Further optimization could improve the performance of the system by testing and adjusting the thresholds, ensuring accurate classification of different sound types.

[0050] It can be seen that the sound tactile feedback system can effectively capture, convert and classify sound signals in the environment, so that the system can generate appropriate feedback according to the sound type. This process includes capturing and amplifying analog signals, analog-to-digital conversion, and classification through thresholds. Optimizing the threshold setting can improve the recognition accuracy of the system, thereby improving the user experience.

[0051] In one embodiment, the model of the receiving chip 22 is but not limited to ESP8266MINI.

[0052] In one embodiment, the receiving chip 22 is connected to the vibration unit 23 via a current control module.

[0053] In one embodiment, see Figure 2The above-mentioned receiving component 20 also includes a battery 24, and the battery 24 is connected to the receiving chip 22 and the vibration unit 23.

[0054] In this embodiment, the captured sound signals undergo preliminary processing steps, including amplification and analog-to-digital conversion, to form digital signals. These signals are then further processed to determine their category and strength; the processed signals are encoded into a structured data packet. This data packet contains information about the classification and strength of the sound. The structure of the data packet may include a signal category identifier (such as the type of audio signal) and a signal strength value (such as the volume).

[0055] The encoded data packets are transmitted from the transmitting component 10 to the receiving component 20 via the ESP-NOW protocol. ESP-NOW is a low-power, low-latency wireless communication protocol that is very suitable for real-time data transmission. This protocol supports point-to-point and point-to-multipoint communication modes, ensuring fast data transmission and synchronization in wireless networks. This communication method can achieve low-latency data transmission and ensure real-time synchronization of tactile feedback, so that users can immediately perceive tactile feedback while receiving audio stimulation.

[0056] In addition, the system is used with headphones or sound systems, and the stimulation received by the user includes audio signals (through headphones or sound systems) and vibrating tactile feedback (from customized wearable devices). This multi-modal perception method is designed to enhance audio stimuli with ambiguous hearing, making the identification of pitch intervals more accurate.

[0057] The receiving component 20 receives the signal from the sound sensor in real time through the ESP-NOW protocol and converts it into tactile feedback. The design of the tactile stimulation takes into account the synchronization with the audio signal to ensure the coordination of the two sensory stimulations.

[0058] Furthermore, users who received multimodal feedback (receiving both auditory and tactile stimulation simultaneously) performed more accurately and consistently on the interval identification task than participants who relied on auditory stimulation alone. The tactile feedback effectively complemented the ambiguity of the audio signal, allowing users to be better supported in identifying intervals.

[0059] In one embodiment, the transmitting and processing chip 12 and the receiving chip 22 of the system of this embodiment are based on the ESP8266 module for data transmission, and the wireless communication capability of ESP8266 is used to realize real-time tactile feedback. The system design takes into account how to efficiently process and transmit data while ensuring the accuracy and timeliness of tactile feedback. This ESP8266-based system has great practical potential, especially in creating an immersive auditory experience for people with hearing impairments. It can not only be used for music learning, but also play a role in other application scenarios that require multimodal perception, such as in education, training and entertainment.

[0060] In one embodiment, the main task of the receiving chip 22 is to process the signal data received from the transmitting end; control the operating state of the vibration motor, and generate corresponding tactile feedback according to the signal strength and type. It includes the following terminals:

[0061] GPIO pin: The GPIO pin is used to control the vibration motor on and off and adjust its vibration state. The high or low level signal of the GPIO pin is used to turn the vibration motor on and off.

[0062] Power pins (VCC and GND): Connect the positive terminal of the battery 24 to the VCC pin of the receiving chip 22, and the negative terminal to the GND pin.

[0063] The following terminals can also be added:

[0064] ADC pin: used to read external sensor data, such as battery 24 voltage, to monitor battery 24 status and ensure that the system is functioning properly.

[0065] In one embodiment, in order to reduce the volume of the vibration unit 23 and ensure that the vibration unit 23 meets basic requirements, a 1034 DC flat-panel mobile phone vibration motor is used as the vibration unit 23, which can generate tactile feedback, adjust the vibration mode according to the strength and type of the received signal, and provide a tactile experience. The two pins of the vibration unit 23 are respectively connected to the GPIO pins of the receiving chip 22. These pins adjust the vibration intensity of the vibration unit 23 through a current control module (such as a MOSFET or a transistor). The vibration intensity and mode of the vibration motor are controlled by a PWM signal, and the working current of the motor is adjusted to change the vibration effect.

[0066] In one embodiment, the battery 24 is but not limited to a polymer soft-pack lithium battery 24. The positive electrode (+) of the battery 24 is connected to the VCC pin of the receiving chip 22 to provide power for the module; the negative electrode (-) of the battery 24 is connected to the GND pin of the receiving chip 22 to form a ground connection for the circuit; ensure that the voltage of the battery 24 is within the working range of the receiving chip 22 (usually 3.3V).

[0067] In one embodiment, the above-mentioned receiving chip 22 and battery 24 are partially placed on the circuit board, connected by connecting wires to ensure the electrical connection between the components and ensure the stable transmission of signals and power; secondly, all components can be installed on a small circuit board, the welding points and connections on the circuit board need to be precise to ensure the reliability of the electrical connection, the circuit board and components need to be fixed in the housing 21, and the design of the housing 21 should take into account the protection function to avoid damage during use, while ensuring that the vibration of the vibration motor can be effectively transmitted to the user's skin. The housing 21 needs to be sturdy and durable, and have good vibration conductivity.

[0068] Through these meticulous designs, the receiving component 20 can efficiently convert signal data into tactile feedback, providing an accurate multi-modal perception experience.

[0069] In one embodiment, see Figure 2 The outer edge of the silicone rubber 25 extends away from the center point of the silicone rubber 25 to form an extension portion.

[0070] The provision of the extension portion can increase the contact area with the human body and improve the stability of the installation of the entire system.

[0071] The system of this embodiment consists of two parts: a transmitting component 10 and a receiving component 20. The transmitting component 10 uses a NodeMCU module to implement core functions, focusing on efficient processing and wireless communication. The receiving component 20 uses an ESP8266MINI module, which is very suitable for wearable applications due to its compact size. In terms of audio input, the system is equipped with a MAX9814 microphone 11, which has high-quality sound capture and automatic gain control, and can respond sensitively to various audio inputs such as weak ambient sounds and human voices. The tactile feedback mechanism is driven by a 1034-type DC flat-panel mobile phone vibration motor, providing tangible and delicate tactile feedback. Communication between devices is achieved through the ESP-NOW protocol, which is designed for short-distance and high-speed data transmission with an effective range of 50 meters, making it very suitable for real-time audio processing and feedback.

[0072] The core functionality of the system relies on the ability to process and interpret sound signals. The system is able to identify significant changes in sound, such as changes in volume or pitch, and distinguish different types of auditory input, such as music or voice, by setting thresholds and responding accordingly. The audio signal is first captured and processed at the transmitting component 10, after which it is converted from an analog signal to a digital signal for more precise operation. After receiving the signal, the receiving component 20 processes it to determine the intensity and characteristics of the sound, and then instructs the vibration motor how to activate to ensure that the tactile feedback accurately reflects the auditory input.

[0073] Therefore, the system of this embodiment can solve the following problems:

[0074] Improved vibration accuracy and resolution: Current wearable music sensing devices can usually only map the high and low notes of music, but have limitations in distinguishing the vibration characteristics of different instruments.

[0075] Improve user comfort and wearability: The device needs to remain comfortable during long-term use. The vibration module in existing devices is usually large. Although the vibration effect is taken into consideration, the wearing comfort and appearance are often ignored.

[0076] Achieve personalized settings: The device needs to adapt to the needs of different users. Current music sensing devices do not fully consider individual differences in vibration perception.

[0077] Reduce the conversion delay from music signals to vibrations: When playing music with fast tempo or obvious beat changes, the vibration feedback of current devices may be out of sync with the music, resulting in an inconsistent user experience.

[0078] In terms of technical solutions, both the transmitting component 10 and the receiving component 20 use the ESP8266 chip. The transmitting component 10 captures the audio signal through the MAX9814 microphone 11 and determines the intensity of the sound by setting a threshold. The signal is processed and sent to the receiving component 20. The receiving component 20 is responsible for further processing the received signal to ensure the effective transmission of the sound signal and send corresponding instructions to the tactile feedback device, thereby ensuring the accuracy and timeliness of the vibration feedback.

[0079] Of course, in addition to being directly used on any part of the body, the system of this embodiment can also be placed in a vest or other clothing for use, and the use effect is as follows: Figure 3 shown.

[0080] The process of the system of this embodiment is mainly presented as wireless communication between two ESP8266 modules using the ESP-NOW protocol. The transmitting component 10 reads the value of the analog sensor. If the value exceeds the set threshold (higher than 550 or lower than 250), it sends a data packet to the receiving component 20. The data packet contains a structure, which contains an integer value (a) and a string (d). In the receiving component 20, after receiving the data, the callback function will parse the data and control the state of the vibration unit 23 according to the received value. If the value of a is 0, the vibration unit 23 presents the first vibration mode and vibration intensity; if the value of a is not 0, the vibration unit 23 presents the second vibration mode and vibration intensity. The purpose of this system is to achieve simple wireless communication between two ESP8266 through the ESP-NOW protocol, and control the vibration unit 23 according to the value of the sensor.

[0081] In one embodiment, the frequency range of the vibration unit 23 is divided into a plurality of frequency bands; and the number of receiving components can be multiple, thereby achieving the purpose of configuring a plurality of vibration units 23 .

[0082] Specifically, the introduction of multi-band vibration units improves the frequency band expression of vibration feedback, enabling it to accurately convey music details in different frequency ranges.

[0083] Implementation:

[0084] Vibration unit division: The system divides the vibration unit into multiple frequency bands according to the frequency range. For example, the low-frequency vibration unit is responsible for processing low-frequency music signals, the medium-frequency vibration unit processes medium-frequency music signals, and the high-frequency vibration unit processes high-frequency music signals.

[0085] Music signal processing: By decomposing the music signal into frequency bands, each vibration unit is ensured to focus on its corresponding frequency range. This segmented processing enables each vibration unit to more accurately convey the music details of a specific frequency band, enhancing the accuracy and expressiveness of the vibration.

[0086] The dynamic vibration intensity adjustment algorithm adopted by this system can adjust the vibration intensity in real time according to the dynamic changes of music, thereby achieving accurate transmission of music details.

[0087] Implementation:

[0088] Intelligent algorithms are used to analyze the dynamic changes of music and automatically adjust the intensity of the vibration unit. For example:

[0089] In the bass part of the music, the system will increase the strength of the vibration unit to ensure a strong sense of low frequency.

[0090] In the high-pitched part of the music, the system will weaken the vibration to avoid excessive vibration affecting the sound quality and details.

[0091] Through real-time analysis and adjustment of music signals, the vibration feedback is ensured to always keep pace with the dynamic changes of the music.

[0092] This system adopts a multi-point tactile feedback design. The comprehensiveness and coverage of tactile feedback can meet the perception needs of different users.

[0093] Implementation:

[0094] Equipped with multiple vibration units, distributed in different parts of the body. For example, back, chest, arms, etc.

[0095] By applying vibrations to multiple parts of the body, the system can more comprehensively convey the rhythm and level changes in the music. This design can avoid the problem that a single vibration position cannot cover all the details of the music and provide a richer tactile experience.

[0096] This system uses vibration feedback synchronization optimization to ensure that the vibration feedback is highly consistent with the audio signal of the music, thereby enhancing the user's music experience.

[0097] Implementation:

[0098] Advanced synchronization optimization technology is used to ensure that the vibration feedback is highly consistent with the key rhythm and strength changes of the music. For example:

[0099] At key rhythm points or changes in volume in music, the system can accurately synchronize vibration feedback to make it completely consistent with the changes in the audio signal.

[0100] Through precise synchronization, users can feel more detailed tactile feedback in complex music clips, thereby improving the overall experience.

[0101] By introducing multi-band vibration units, applying dynamic vibration intensity adjustment algorithms, designing multi-point tactile feedback systems, and optimizing vibration feedback synchronization, the tactile feedback system will be significantly improved in terms of vibration intensity and accuracy. These measures will effectively enhance the transmission of music details, allowing users to enjoy a more realistic and delicate music experience.

[0102] The system of this embodiment converts audio signals into tactile sensations. This process first captures music signals through advanced sound sensors, and then uses complex algorithms to convert these audio signals into precise vibration patterns, ensuring that the vibrations are highly synchronized and consistent with the original audio signals.

[0103] In this embodiment, the key to converting audio signals into tactile perception lies in the design of the audio signal processing algorithm. The purpose of this algorithm is to ensure that the vibration pattern is highly consistent with the rhythm and frequency of the music signal to enhance the user experience. The following is a detailed description of the specific algorithm processing steps:

[0104] Step 1: Audio signal acquisition and preprocessing:

[0105] Acquisition: First, use advanced sound sensors to capture audio signals. High-quality acquisition equipment can ensure the accuracy and integrity of the signal.

[0106] Preprocessing: Before further analysis of the audio signal, the signal is preprocessed. This includes:

[0107] Denoising: Remove background noise through filters to improve the purity of the signal.

[0108] Standardization: Standardize the amplitude of the audio signal to ensure that the signal strength is within a reasonable range and avoid the effects of being too strong or too weak.

[0109] Step 2: Frequency decomposition:

[0110] Spectral analysis: Decomposing audio signals into different frequency components using methods such as Fast Fourier Transform (FFT) or Wavelet Transform. These methods can decompose complex audio signals into basic frequency components.

[0111] Frequency band division: According to the frequency characteristics of the audio signal, it is divided into three main frequency bands: low frequency, medium frequency and high frequency. The processing method of each frequency band can be different to better reflect the different elements of the music.

[0112] Step 3: Rhythm Analysis:

[0113] Beat Detection: Analyzes the rhythm of audio signals using Short-Time Fourier Transform (STFT) techniques, including identifying beat points in music.

[0114] Volume changes: Analyze changes in volume so that changes in the volume of the music are reflected in the vibration pattern. This helps synchronize the rhythm and volume of the vibration pattern.

[0115] To ensure that the vibration pattern is synchronized with the rhythm of the music, the algorithm performs rhythm analysis on the signal, including beat detection and analysis of volume changes. This part of the processing uses short-time Fourier transform (STFT) technology to identify the beat points and changes in strength in the music, thereby dynamically adjusting the rhythm and intensity of the vibration.

[0116] Step 4: Vibration pattern generation:

[0117] Pattern Design: Generate vibration patterns based on the results of frequency decomposition and rhythm analysis. Each frequency band corresponds to a specific vibration pattern. For example, a low frequency band may correspond to a strong vibration, while a high frequency band may correspond to a more delicate vibration.

[0118] Mapping function: Map the volume to the intensity of vibration. The louder the volume, the stronger the vibration. For different frequencies, generate corresponding vibration frequencies.

[0119] After completing the frequency decomposition and rhythm analysis, the algorithm will generate the vibration patterns of different vibration units based on the comprehensive information of frequency, rhythm and volume. The louder the volume, the greater the vibration intensity, which is achieved through the mapping function from volume to vibration intensity. For audio signals of different frequencies, the algorithm converts them into corresponding vibration frequencies.

[0120] Step 5: Multi-point tactile feedback coordination:

[0121] For systems with multiple vibration units, the algorithm coordinates the vibration modes of different units in real time to ensure that vibrations at different locations are synchronized in time and form a suitable tactile experience in space. For example, strong vibrations in the low-frequency band can be concentrated in a certain area of ​​the user's body, while subtle vibrations in the high-frequency band can be distributed in multiple areas to provide a richer sense of tactile hierarchy.

[0122] Step 6: Real-time adjustment and optimization:

[0123] Dynamic processing: Music signals change over time, and the algorithm needs to be able to process them in real time. Through deep learning methods such as convolutional neural networks (CNN), the algorithm can predict the changing trend of audio signals and adjust the vibration mode in advance.

[0124] Feedback Adjustment: Based on the prediction results, the vibration pattern is adjusted in real time to ensure that the tactile feedback is consistent with the changes in the music signal.

[0125] Music signals tend to change dynamically over time, so the algorithm must have real-time processing capabilities. Through deep learning methods such as convolutional neural networks (CNN), the algorithm can learn and predict the changing trends of audio signals, adjust the vibration mode in advance, and synchronize the tactile feedback with the changes in the music signal.

[0126] Step 7: Personalization settings:

[0127] User Adjustment: The system allows users to personalize the vibration mode according to their personal hearing and tactile preferences. This includes adjusting parameters such as vibration intensity, frequency range and vibration location to optimize the user experience.

[0128] Save settings: Users can save customized vibration patterns for future use.

[0129] The system allows users to personalize the vibration mode. By adjusting the parameters of the vibration mode, such as intensity threshold, frequency range and vibration position, users can adjust it according to their personal hearing and tactile preferences to ensure the best experience.

[0130] Step 8: Fine-tune the vibration motor:

[0131] The device uses high-performance vibration motors that not only need to respond quickly, but also be able to produce vibrations of various intensities and frequencies to accurately convey different elements of music, such as rhythm, pitch and intensity.

[0132] Step 9: Wireless connection technology:

[0133] Using Bluetooth or other wireless technologies, the system can seamlessly connect with various music playback devices, providing a more convenient and flexible user experience.

[0134] The system is designed with ergonomics in mind, using skin-friendly and soft materials and a compact hardware layout to ensure user comfort during extended wear, while enabling vibrations to be effectively transmitted to the user's sensitive areas.

[0135] The system integrates sound-to-tactile conversion technology into a wearable device, taking into account both technical performance and appearance design, while also considering weight and portability.

[0136] The system's interactive design is simple and intuitive, especially to meet the needs of the hearing-impaired, ensuring that it is easy to operate and understand.

[0137] The system is able to provide personalized vibration modes, adjusting the vibration intensity and pattern according to the user's personal preferences and needs.

[0138] The above-mentioned sound tactile feedback system is provided with a transmitting component 10 and a receiving component 20, wherein the transmitting component 10 includes a microphone 11 for collecting sound signals in the external environment, and the transmitting processing chip 12 classifies and encodes the sound signals collected by the microphone 11 to form data packets, and sends these data packets to the receiving component 20; the receiving component 20 includes a shell 21 for accommodating a receiving chip 22 and a vibration unit 23, one end of the shell 21 is provided with an opening and is covered with silicone 25, the receiving chip 22 is used to receive the data packet sent by the transmitting component 10 and generate a corresponding vibration signal, the vibration signal includes a vibration mode and a vibration intensity. The vibration unit 23 is used to work according to the received vibration signal, generate tactile feedback, provide a corresponding vibration experience, and realize easy carrying and use. It can be placed in a user-specified location for use according to the user's personalized needs to meet the special needs of different users.

[0139] In one embodiment, see Figure 4 , also provides a working method performed by the above-mentioned sound tactile feedback system, including steps S110 to S140.

[0140] S110, the user sticks the receiving component 20 to any position on the body, and the microphone 11 collects sound signals in the external environment;

[0141] S120 , the transmitting processing chip 12 classifies the sound signal collected by the microphone 11 into sound types, encodes them into corresponding data packets, and sends them to the receiving component 20 .

[0142] In this embodiment, the transmitting processing chip 12 classifies the sound type of the sound signal collected by the microphone 11 according to a set signal threshold, encodes it into a corresponding data packet, and sends it to the receiving component 20.

[0143] S130, the receiving chip 22 receives the data packet, and generates a corresponding vibration signal according to the data packet, wherein the vibration signal includes a vibration mode and a vibration intensity;

[0144] S140 : The vibration unit 23 operates according to the vibration signal to generate tactile feedback.

[0145] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the working method of the above-mentioned sound tactile feedback system can refer to the corresponding description in the aforementioned sound tactile feedback system embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0146] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A sound tactile feedback system, characterized in that: include: A transmitting component and a receiving component, wherein the transmitting component includes a microphone and a transmitting processing chip, and the microphone is used to collect sound signals in the external environment; The transmitting processing chip is used to classify the sound signal collected by the microphone into sound types, encode it into corresponding data packets, and send it to the receiving component; the receiving component includes a shell, a receiving chip and a vibration unit, the receiving chip and the vibration unit are built in the shell, one end of the shell is provided with an opening, the vibration unit is located at the opening, and the opening is covered with silica gel; the receiving chip is used to receive the data packet and generate a corresponding vibration signal according to the data packet, wherein the vibration signal includes a vibration mode and a vibration intensity; The vibration unit is used to work according to the vibration signal to generate tactile feedback.

2. The sound tactile feedback system according to claim 1, characterized in that: The model of the microphone is MAX9814.

3. The sound tactile feedback system according to claim 1, characterized in that: The transmitting processing chip sends the data packet to the receiving chip through the ESP-NOW protocol.

4. The sound tactile feedback system according to claim 1, characterized in that: The transmitting processing chip is used to classify the sound type of the sound signal collected by the microphone according to the set signal threshold, encode it into corresponding data packets, and send it to the receiving component.

5. The sound tactile feedback system according to claim 1, characterized in that: The model of the receiving chip is ESP8266MINI.

6. The sound tactile feedback system according to claim 5, characterized in that: The receiving chip is connected to the vibration unit through a current control module.

7. The sound tactile feedback system according to claim 6, characterized in that: The receiving component also includes a battery, and the battery is connected to the receiving chip and the vibration unit.

8. The sound tactile feedback system according to claim 1, characterized in that: The outer edge of the silicone rubber extends in a direction away from the center point of the silicone rubber to form an extension portion.

9. A working method performed by the sound tactile feedback system according to any one of claims 1 to 8, characterized in that: include: The user sticks the receiving component anywhere on the body, and the microphone collects sound signals from the external environment; The transmitting processing chip classifies the sound signal collected by the microphone into sound types, encodes it into corresponding data packets, and sends it to the receiving component; The receiving chip receives the data packet and generates a corresponding vibration signal according to the data packet, wherein the vibration signal includes a vibration mode and a vibration intensity; The vibration unit operates according to the vibration signal to generate tactile feedback.

10. The working method of the sound tactile feedback system according to claim 9, characterized in that: The transmitting processing chip classifies the sound signal collected by the microphone into sound types, encodes it into corresponding data packets, and sends it to the receiving component, including: The transmitting processing chip classifies the sound signal collected by the microphone according to the set signal threshold, encodes it into a corresponding data packet, and sends it to the receiving component.

Images