Multi-mode visual touch music perception garment

Abstract

The invention provides a multi-mode visual touch music sensing garment. The multi-mode visual touch music sensing garment comprises a garment body, and a sound sensor, an audio analysis module, a main processor, a plurality of lamp strips and at least one vibration motor which are arranged on the garment body, the sound sensor is used for collecting analog audio signals; the audio analysis module is used for performing spectral analysis on the analog audio signal, dividing the analog audio signal into a plurality of frequency bands and transmitting the frequency bands to the main processor; the main processor is used for selecting frequency values of different frequency bands and respectively mapping the frequency values to different lamp strips, and controlling the corresponding lamp strips to flicker along with rhythm; the vibration frequency is output according to the received audio signals of the multiple frequency bands, the corresponding vibration motor is controlled to vibrate according to the vibration frequency and the vibration intensity, and the vibration intensity changes along with the change of the amplitude of the audio signals. According to the music perception garment provided by the invention, music visualization and vibration touch are realized, multi-mode design and garment design are fused, and hearing-impaired people are helped to feel rhythm and wonderness of music in an immersive manner.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent clothing, and in particular relates to a multi-modal visual-tactile music perception clothing. Background Art

[0002] As a large group, the hearing-impaired people encounter inconveniences when facing music activities that require hearing, which makes this group's social identity and communication expectations generally low.

[0003] Touch, like hearing, is a very important form of perception. Human touch is a very complex sense, and its sensory organs are all over the body, including rich perceptual information such as touch, pressure, and vibration, and can well perceive special or complex information expressed in addition to the auditory form. Early studies have shown that hearing-impaired people are more sensitive to vibration touch than normal people. Music visualization design is a means of realizing visual compensation for hearing. The important reason why music visualization can be realized is that there are a large number of parameters in music. Visualization design is to image each parameter in music and map different visual attributes through music rhythm speed and volume. From the perspective of the hearing-impaired, they are eager to experience music. Researchers have found that regardless of the type and degree of hearing impairment, they all show a desire and need for music experience. How to meet their needs has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve all or part of the above problems, the present invention aims to provide a multimodal visual-tactile music perception clothing.

[0005] According to one aspect of the present invention, a multimodal visual-tactile music perception clothing is provided, comprising: a clothing body suitable for being worn on the body, and a sound sensor, an audio analysis module, a main processor, a plurality of light strips, and at least one vibration motor arranged on the clothing body; the sound sensor is connected to the audio analysis module through a wire, and the main processor is respectively connected to the audio analysis module, the plurality of light strips, and the at least one vibration motor through a wire;

[0006] The sound sensor is used to collect analog audio signals;

[0007] The audio analysis module is used to perform spectrum analysis on the analog audio signal, divide it into multiple frequency bands and transmit it to the main processor;

[0008] The main processor is used to select frequency values ​​of different frequency bands and map them to different light strips respectively, and control the corresponding light strips to flash with the rhythm; and output the vibration frequency according to the received audio signals of multiple frequency bands, and control the corresponding vibration motor to vibrate according to the vibration frequency and vibration intensity, wherein the vibration intensity changes with the change of the amplitude of the audio signal.

[0009] Optionally, the number of the vibration motors is four;

[0010] The main processor is specifically used to: determine a first audio segment and a second audio segment according to the received audio signals of multiple frequency bands, wherein the frequency of the second audio segment is higher than the frequency of the first audio segment; and respectively output a vibration frequency corresponding to the first audio segment and a vibration frequency corresponding to the second audio segment, control two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the first audio segment, and control the other two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the second audio segment.

[0011] Optionally, the vibration frequency corresponding to the first audio segment is determined based on the difference between the maximum amplitude of the audio signal of the first audio segment and a first threshold; the vibration frequency corresponding to the second audio segment is determined based on the difference between the maximum amplitude of the audio signal of the second audio segment and a second threshold.

[0012] Optionally, the audio analysis module is specifically used to: perform spectrum analysis on the analog audio signal to divide it into multiple frequency bands; strengthen or weaken the signal in a specific frequency band through a built-in graphic equalizer filter; and rectify the signal in each frequency band, calculate the average value, and then pass it to the main processor.

[0013] Optionally, each light strip has n LED lights, where n is greater than 1;

[0014] The main processor is specifically used to select frequency values ​​of different frequency bands and map them to different light strips respectively, and control different LED lights of the corresponding light strips to flash according to different ranges reached by the audio signal amplitude.

[0015] Optionally, the sound sensor is fixed under the lapel of the clothing body; and / or the audio analysis module and the main processor are arranged in the back interlayer of the clothing body; and / or the multiple light strips are arranged on both sides of the chest of the clothing body; and / or the at least one vibration motor is arranged on the clothing body at a position corresponding to the vibration sensing part of the body.

[0016] Optionally, the two vibration motors are arranged on the clothing body at positions corresponding to clavicle parts of the body, and the other two vibration motors are arranged on the clothing body at positions corresponding to wrist parts of the body.

[0017] Optionally, the at least one vibration motor is a button-type vibration motor.

[0018] Optionally, the outer fabric of the clothing body is PU leather and translucent frosted PVC fabric, and the inner fabric of the clothing body is fabric with a waterproof coating.

[0019] It can be seen from the above technical solution that the multimodal visual-tactile music perception clothing provided by the present invention has the following beneficial effects:

[0020] The present invention combines an audio analysis module, a light strip, a vibration motor and a main processor, performs spectrum analysis on the collected audio signals, and maps the values ​​of different frequency bands to the light strip and the vibration motor, thereby realizing music visualization and vibration touch, and integrating multimodal design with clothing design. The music perception clothing can help the hearing-impaired to truly see and feel the music, actively interact with the music, and establish an emotional connection, thus helping the hearing-impaired to enrich and develop their music experience in an all-round and in-depth manner. The music perception clothing not only meets the functional needs, but also fully reflects humanistic care. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of the components of the multimodal visual-tactile music perception clothing provided by an embodiment of the present invention;

[0022] Figure 2 A physical picture of a sound sensor according to an embodiment of the present invention;

[0023] Figure 3a is a physical diagram of an audio analysis module in an embodiment of the present invention;

[0024] Figure 3b A pin diagram of an audio analysis module in an embodiment of the present invention;

[0025] Figure 4a A physical diagram of a vibration motor in an embodiment of the present invention;

[0026] Figure 4b This is a diagram showing the internal structure of a vibration motor in an embodiment of the present invention;

[0027] Figure 5 A physical diagram of a main processor in an embodiment of the present invention;

[0028] Figure 6 Schematic diagram of circuit connection of each component module in an embodiment of the present invention;

[0029] Figure 7 A flowchart of music rhythm visualization in an embodiment of the present invention;

[0030] Figure 8 This is a flow chart of vibrotactile representation of music rhythm in an embodiment of the present invention;

[0031] Fig. 9 An example diagram of styles of multimodal visual-tactile-music-perception clothing provided by an embodiment of the present invention;

[0032] Fig.10A physical display diagram of the multimodal visual-tactile music perception clothing provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to better understand the purpose, structure and function of the present invention, a breathing visualization clothing of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0034] Figure 1 A schematic diagram of the components of the multimodal visual-tactile music perception clothing provided by an embodiment of the present invention, such as Figure 1 As shown, the multimodal visual-tactile music perception clothing includes, in addition to a clothing body suitable for being worn on the body, also includes: a sound sensor, an audio analysis module, a main processor, a plurality of light strips and at least one vibration motor arranged on the clothing body; the sound sensor is connected to the audio analysis module through a wire, and the main processor is respectively connected to the audio analysis module, the plurality of light strips and at least one vibration motor through a wire.

[0035] The sound sensor is used to collect analog audio signals; the audio analysis module is used to perform spectrum analysis on the analog audio signals, divide them into multiple frequency bands and transmit them to the main processor; the main processor is used to select frequency values ​​of different frequency bands and map them to different light strips respectively, and control the corresponding light strips to flash with the rhythm; and, output vibration frequencies according to the received audio signals of multiple frequency bands, and control the corresponding vibration motors to vibrate according to the vibration frequency and vibration intensity, wherein the vibration intensity changes with the change of the amplitude of the audio signal.

[0036] This embodiment uses a large number of parameters in music to perform mapping processing to achieve different visual attributes and vibration effects, thereby realizing the combination of music visualization and skin touch to experience music. The technical design of multimodal intelligent perception clothing is applied to the leisure and entertainment assistance of the hearing-impaired, helping the hearing-impaired to perceive and experience music, generate a pleasant and immersive feeling, effectively establish emotional connections, enhance a sense of identity, and obtain a better music experience.

[0037] Figure 2 This is a physical picture of the sound sensor in the embodiment of the present invention. In this embodiment, the sound sensor is specifically a microphone-based sound detection sensor, which is intended to measure the sound intensity in the surrounding environment. The sensor is equipped with a 300-fold amplifier, which can perform AD acquisition under 3.3V and 5V voltage references by outputting analog audio signals. The circuit connection method of the sound sensor is very simple. It only needs to connect the positive power supply, ground terminal and analog signal port of the control board. Its main application areas include achieving interactive effects based on sound intensity, making voice-controlled robots, designing voice-controlled switches, and applying to voice-controlled alarm systems and other scenarios. Figure 2 Detailed description of the invention FIG. 4 is a physical diagram of a sound sensor in an embodiment of the present invention.

[0038] Figure 3a is a physical diagram of an audio analysis module in an embodiment of the present invention, Figure 3b Pin diagram of the audio analysis module in the embodiment of the present invention. In the present embodiment, the audio analysis module is specifically an audio signal analysis module, which is designed based on the MSGEQ7 graphic equalizer filter, has a very low quiescent current (less than 1 mA), and a power supply voltage range of 2.7 to 5.5V, wherein the best performance can be achieved under the 5V power supply state, and is suitable for portable audio devices. The left and right input ports on the right side of the module are used to connect the sound sensor, and the left port is mainly used for signal output, power supply, and reset and gating control. Its working principle is mainly to collect analog audio signals (specifically analog voltage signals) through sound sensors (such as microphone sensors or audio signal connectors), perform spectrum analysis on the analog audio signals, and divide them into multiple frequency bands; strengthen or weaken the signals of specific frequency bands through the built-in graphic equalizer filter; and, rectify the signals in each frequency band, calculate the average value, and then pass them to the main processor.

[0039] In a specific example, the audio analysis module divides the input signal into seven frequency bands preset inside the IC, namely 63Hz, 160Hz, 400Hz, 1KHz, 2.5kHz, 6.25kHz and 16kHz; then the signal is strengthened or weakened by the built-in graphic equalization filter of MSGEQ7 to make the amplitude of each frequency band easier to measure and display; in each frequency band, the signal is first rectified and then the average value is calculated to reduce noise and provide stable output.

[0040] In an optional embodiment, the light strip is an RGB LED color light strip, specifically a lighting device composed of a series of full-color LEDs, and only one pin is needed to control all LEDs. Each LED is an independent pixel, and each pixel contains three primary colors: red (R), green (G) and blue (B), which can achieve 0-256 levels of brightness display, thereby presenting 16777216 true colors. Each pixel integrates an intelligent digital interface, a data latch signal, and a shaping and amplifying drive circuit, and also includes a signal shaping circuit, which effectively ensures the high consistency of the light color of the pixel. The input voltage of the light strip is 5V, and the power consumption of each LED lamp is 0.2W, which has low energy consumption. The display color is pure, with great flexibility, and can be easily connected, bent and cut. And it is equipped with 3M adhesive tape on the back, which can be easily fixed on clothing.

[0041] In the embodiment of the present invention, in order to map the changes of music to different visual attributes, the main processor is used to select the frequency values ​​of different frequency bands and map them to different light strips respectively, and control the corresponding light strips to flash with the rhythm. Therefore, the number of light strips is the same as the number of frequency bands. Corresponding to the above example, the number of light strips is seven, and the seven frequency bands with different frequencies output by the audio analysis module correspond to seven light strips respectively. Each light strip has n LED lights, where n is greater than 1. For example, if n is 10, each light strip has 10 LED lights, connected into 7 × 10 rectangular matrix, forming a snake effect. Set the RGB values ​​of the 10 LED lights on each light strip to (0,200,0), (0,160,40), (0,120,80), (0,80,120), (0,40,160), (0,0,200), (40,0,160), (80,0,120), (120,0,80), (160,0,0). After the main processor processes the mapping, it controls the seven LED lights with different R, G, and B values ​​to flash according to the rhythm.

[0042] In the embodiment of the present invention, in order to further achieve a more delicate rhythm expression effect, at least one vibration motor is also used on the clothing to produce different vibration effects in combination with music changes. Figure 4a This is a physical picture of the vibration motor in the embodiment of the present invention. Figure 4b FIG. 1 is a diagram showing the internal structure of a vibration motor in an embodiment of the present invention. Figure 4a and Figure 4b As shown, the vibration motor is a button-type vibration motor. Specifically, the vibration motor adopts a mini button-type vibration motor with a diameter of 1 cm and a net weight of about 0.8 grams. It is small and light, has sufficient vibration force, and the vibration mode is up and down vibration. It has low power consumption, fast response, and no long-term delay. In the wide range of rated voltage 4.2v, the vibration intensity will increase with the increase of output voltage.

[0043] Figure 5This is a physical picture of the main processor in the embodiment of the present invention. In this embodiment, the main processor adopts an Arduino-compatible microprocessor, namely the Arduino Mega2560 controller. The processor is small in size, only 58cm×34cm in size, and has low power consumption. It is one of the better choices for wearable devices. The processor has a total of 2 groups of power ports, 54 digital ports (16 of which can be used for PWM input), 16 analog inputs, 4 UART interfaces, a 16MHz crystal oscillator, a USB port, a power socket, an ICSP header and a reset button. Arduino Mega2560 is also compatible with the expansion board designed for Arduino UNO. Two pins SDA and SCL are added at AREF to support the I2C interface and add IOREF. Its input voltage is 6~8V, and the operating voltage is 5V. The program can be directly downloaded and debugged through the Micro USB interface.

[0044] Figure 6 This is a schematic diagram of the circuit connection of each component module in the embodiment of the present invention. Figure 6 As shown, no further details are given.

[0045] Based on the above description of the hardware part, the functions of the software part of the embodiment of the present invention are summarized as follows: the audio analysis module transmits the signals of multiple frequency bands to the main processor, and the main processor uses the software program to map the frequency values ​​of different frequency bands to different light strips, and controls the corresponding light strips to flash with the rhythm. At the same time, the main processor uses the software program to calculate and output the vibration frequency, and controls the corresponding vibration motor to vibrate according to the vibration frequency and vibration intensity, wherein the vibration intensity changes with the change of the amplitude of the audio signal. With this function, the effect of visual and vibrotactile simultaneously representing the rhythm of the music is achieved.

[0046] In an optional implementation, the music rhythm visualization is specifically as follows: the main processor selects frequency values ​​of different frequency bands and maps them to different light strips respectively, and controls the flashing of different LED lights of the corresponding light strips according to the different ranges reached by the audio signal amplitude.

[0047] Figure 7 FIG. 1 is a flowchart of music rhythm visualization in an embodiment of the present invention. Figure 7 As shown, after starting, the program is initialized first, and then the light strip is initialized and the snake effect is set; the microphone sensor (i.e., sound sensor) and the audio analysis module recognize the audio signal, define i audio bands (also called frequency bands), where 0≤i<7, read the audio signal amplitude of each frequency band, and form an array l[i]; 10 amplitude ranges are pre-set, and different LED lights of the corresponding light strip are controlled to flash according to different ranges reached by the audio signal amplitude.

[0048] In the following example, the system divides the input music frequency into seven frequency bands of 63Hz, 160Hz, 400Hz, 1KHz, 2.5kHz, 6.25kHz and 16kHz through the audio analysis module, refers to various music types and instruments, and combines the sensor recognition function to divide the first two frequency bands of 63Hz and 160Hz into the basis for the characterization of low-frequency bands, and the latter frequency bands of 400Hz, 1KHz, 2.5kHz, 6.25kHz and 16kHz into the basis for the characterization of high-frequency bands.

[0049] In an optional embodiment, four vibration motors are set as an example for explanation, wherein two vibration motors are used to characterize the rhythm of music in the low frequency band, and the other two vibration motors are used to characterize the rhythm of music in the high frequency band. The vibration tactile characterization of the rhythm of music is specifically as follows: determining the first audio band and the second audio band according to the received audio signals of multiple frequency bands, wherein the frequency of the second audio band is higher than the frequency of the first audio band, and the first audio band is a frequency band in the low frequency band, for example, the first audio band is selected as 160Hz, and the second audio band is a frequency band in the high frequency band, for example, the second audio band is selected as 6.25kHz; and, outputting the vibration frequency corresponding to the first audio band and the vibration frequency corresponding to the second audio band respectively, controlling the two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the first audio band, and controlling the other two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the second audio band. The vibration frequency corresponding to the first audio band is determined according to the difference between the maximum amplitude of the audio signal of the first audio band and the first threshold value; the vibration frequency corresponding to the second audio band is determined according to the difference between the maximum amplitude of the audio signal of the second audio band and the second threshold value.

[0050] Figure 8 FIG. 1 is a flow chart of vibrotactile representation of music rhythm in an embodiment of the present invention. Figure 8 As shown, after starting, the program is initialized first; the microphone sensor (i.e., sound sensor) and the audio analysis module identify the audio signal, define the i-segment audio band (also called frequency band), where 0≤i<7, assign the variable M to the maximum amplitude of the i-segment audio signal, if i=1 (frequency band of 160Hz, i.e., the first audio band), then calculate the bass vibration frequency=M-200, where 200 is the above-mentioned first threshold value, if i=5 (frequency band of 6.25kHz, i.e., the second audio band), then calculate the treble vibration frequency=M-50, where 50 is the above-mentioned second threshold value. After a delay of 0.1s, output the bass vibration frequency to the two vibration motors in the bass frequency band, control the two vibration motors in the bass frequency band to vibrate according to the bass vibration frequency and vibration intensity, output the treble vibration frequency to the two vibration motors in the treble frequency band, control the two vibration motors in the treble frequency band to vibrate according to the treble vibration frequency and vibration intensity.

[0051] The present invention is not limited to setting four vibration motors, and any number of vibration motors can be set, each vibration motor corresponds to a frequency band, and when an audio signal of the corresponding frequency band is received, the vibration frequency is calculated and output, and the corresponding vibration motor is controlled to vibrate according to the vibration frequency and vibration intensity. The present invention does not limit the number of vibration motors.

[0052] Regarding the design of the combination of the above-mentioned electronic devices and the clothing body, the embodiments of the present invention may adopt the following design methods: the sound sensor is fixed under the lapel of the clothing body; and / or, the audio analysis module and the main processor are arranged in the back interlayer of the clothing body; and / or, multiple light strips are arranged on both sides of the chest of the clothing body; and / or, at least one vibration motor is arranged on the clothing body at a position corresponding to the vibration sensing part of the body. Furthermore, one vibration motor is arranged on the clothing body at a position corresponding to the clavicle part of the body, and another vibration motor is arranged on the clothing body at a position corresponding to the wrist part of the body.

[0053] In terms of visual representation, the chest area generally floats slightly. Placing the LED light strips on both sides of the chest allows the flashing and color changes of the LED light strips to be more intuitively observed. In addition, experiments show that the wrist area is most sensitive to vibration tactile perception. Placing the vibration motor in the high-frequency band at the wrist of the clothing can better allow the wearer to perceive the corresponding rhythm intensity. When the high-frequency sound appears intermittently or continuously, it can be keenly perceived by the wearer. The vibration motor in the low-frequency band is placed at the clavicle. Studies have shown that, unlike other representative parts, the vibration transmission at the clavicle can produce a more immersive experience for the wearer. Taking four vibration motors as an example, two of the vibration motors are set on the clothing body at the position corresponding to the clavicle area of ​​the body (one vibration motor is symmetrically set on each of the left and right clavicles), and the other two vibration motors are set on the clothing body at the position corresponding to the wrist area of ​​the body (one vibration motor is symmetrically set on each of the left and right wrists).

[0054] Fig. 9 This is an example diagram of the styles of multi-modal visual-tactile-music-perception clothing provided by an embodiment of the present invention. Fig.10 A physical display diagram of a multimodal visual-tactile music perception clothing provided by an embodiment of the present invention, such as Fig. 9 and Fig.10 As shown in the figure, the music sensing clothing is a women's casual suit jacket. The outer fabric of the clothing body is PU leather and translucent frosted PVC fabric. PU leather is stiffer and has stronger covering power than ordinary fabrics. It is used for sleeves, collars and wrinkle-free parts of the main body; translucent frosted PVC fabric is used for the wrinkle design parts; the inner fabric of the clothing body is a fabric with a waterproof coating. All three are waterproof fabrics. While assisting in protecting the circuit safety, they also solve some cleaning problems. When stains appear on the surface of the clothing, they can be wiped locally.

[0055] In terms of design, the PVC fabric, pleated design, and sheet light strips on the chest work together to make the flashing lights dazzling. The vibration motor and wires in the high-frequency band are fixed to the outer layer and inner layer of the sleeves, the sound sensor is fixed under the collar, the vibration motor in the low-frequency band is fixed at the clavicle, and the Arduino system processor is placed on the back. Due to the translucent nature of the PVC frosted fabric on the back, a double-layer lining is set on the back, and the complete circuits are hidden in the double-layer lining on the back or in the lining and outer layer stitched together to achieve an invisible effect. The battery is placed in the right pocket, which is portable and detachable.

[0056] Based on the solution provided by the present invention, a real-person fitting experiment is carried out in a normal environment. The experimenter imitates the hearing-impaired state of hearing loss, and inserts sleeping earplugs into both ears to reduce the auditory perception of the experimenter. The main function of music perception clothing is to assist hearing-impaired people to understand and perceive music. Therefore, after wearing music perception clothing, the experimenter starts to randomly play music from the music library. In order to ensure the accuracy of the test audio analysis module, the music library should cover common music types on the market. The music types can be divided into two modules: vocal music and pure music. The music library selects vocal music with lyric, jazz and rock styles, and selects pure music in the form of piano, guitar, electronic music and other forms.

[0057] When the music starts playing, the LED lights on the chest will rise and fall synchronously with the rhythm of the music, and the wrists and collarbones can produce vibration tactile sensations, and can clearly distinguish between high-frequency and low-frequency transmission vibrations. The performance effects of vocal music and pure music can both assist the experimenters in listening, and are not restricted by the style of music and the form of performance. They have a good visual and tactile experience, allowing the experimenters to experience the rhythm of the music and the high and low frequency changes of the music when their hearing is impaired, and further experience and understand the emotional flow conveyed by the music. The experiment shows that the designed multimodal visual-tactile music perception clothing can help the hearing-impaired people immersively feel the rhythm and beauty of music, which meets the design requirements.

[0058] In order to solve the problem of low perception, poor autonomy and weak sense of identity of hearing-impaired people in participating in music activities, the present invention combines audio analysis module, RGB LED light strip, button-type vibration motor and Arduino system to design a product that can help hearing-impaired people to really see music, feel music, actively interact with music and have emotional connection, so as to enrich and develop the music experience of hearing-impaired people in an all-round and in-depth manner. The music perception clothing not only meets the functional needs, but also fully reflects humanistic care.

[0059] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0060] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention. In particular, as long as there is no structural conflict, the various technical features mentioned in each embodiment can be combined in any way. The present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A multimodal visual-tactile music perception clothing, characterized in that: include: A garment body suitable for being worn on the body, and a sound sensor, an audio analysis module, a main processor, a plurality of light strips, and at least one vibration motor arranged on the garment body; the sound sensor is connected to the audio analysis module through a wire, and the main processor is respectively connected to the audio analysis module, the plurality of light strips, and the at least one vibration motor through a wire; The sound sensor is used to collect analog audio signals; The audio analysis module is used to perform spectrum analysis on the analog audio signal, divide it into multiple frequency bands and transmit it to the main processor; The main processor is used to select frequency values ​​of different frequency bands and map them to different light strips respectively, and control the corresponding light strips to flash with the rhythm; and output the vibration frequency according to the received audio signals of multiple frequency bands, and control the corresponding vibration motor to vibrate according to the vibration frequency and vibration intensity, wherein the vibration intensity changes with the change of the amplitude of the audio signal.

2. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: The number of the vibration motors is four; The main processor is specifically used to: determine a first audio segment and a second audio segment according to the received audio signals of multiple frequency bands, wherein the frequency of the second audio segment is higher than the frequency of the first audio segment; and respectively output a vibration frequency corresponding to the first audio segment and a vibration frequency corresponding to the second audio segment, control two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the first audio segment, and control the other two vibration motors to vibrate according to the vibration frequency and vibration intensity corresponding to the second audio segment.

3. The multimodal visual-tactile music perception clothing according to claim 2, characterized in that: The vibration frequency corresponding to the first audio segment is determined according to the difference between the maximum amplitude of the audio signal of the first audio segment and a first threshold; the vibration frequency corresponding to the second audio segment is determined according to the difference between the maximum amplitude of the audio signal of the second audio segment and a second threshold.

4. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: The audio analysis module is specifically used to: perform spectrum analysis on the analog audio signal to divide it into multiple frequency bands; enhance or weaken the signal of a specific frequency band through a built-in graphic equalization filter; and rectify the signal in each frequency band, calculate the average value, and then pass it to the main processor.

5. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: Each light strip has n LED lights, where n is greater than 1; The main processor is specifically used to select frequency values ​​of different frequency bands and map them to different light strips respectively, and control different LED lights of the corresponding light strips to flash according to different ranges reached by the audio signal amplitude.

6. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: The sound sensor is fixed under the lapel of the clothing body; and / or, the audio analysis module and the main processor are arranged in the back interlayer of the clothing body; and / or, the multiple light strips are arranged on both sides of the chest of the clothing body; and / or, the at least one vibration motor is arranged on the clothing body at a position corresponding to the vibration sensing part of the body.

7. The multimodal visual-tactile music perception clothing according to claim 2, characterized in that: The two vibration motors are arranged on the clothing body at positions corresponding to the clavicle parts of the body, and the other two vibration motors are arranged on the clothing body at positions corresponding to the wrist parts of the body.

8. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: The at least one vibration motor is a button-type vibration motor.

9. The multimodal visual-tactile music perception clothing according to claim 1, characterized in that: The outer fabric of the clothing body is PU leather and translucent frosted PVC fabric, and the inner fabric of the clothing body is a fabric with a waterproof coating.

Images