Alternative sensory delivery system

Abstract

The present invention relates to an alternative sensory delivery system comprising: a motion detection unit for detecting the movement of a dancer; a conversion unit for converting information detected by the motion detection unit into tactile information; and a haptic unit for implementing, into a tactile signal for a recipient, the tactile information converted by the conversion unit.

Description

Alternative Sensory Transmission System

[0001] The present invention relates to an alternative sensory delivery system, and more specifically, to an alternative sensory delivery system that reduces the gap in dance learning and appreciation between hearing- and visually-impaired people and the general public by converting audiovisual information into tactile information and delivering it to the user.

[0002] Typically, enjoying cultural performances or learning about the arts and physical education is achieved through audiovisual means. Therefore, those with impaired or impaired hearing or visual abilities may find it difficult to enjoy cultural performances or learn the movements required for the arts and physical education. Typically, audio commentary is structured around the general audience, making it difficult for those with hearing or visual impairments to understand. While sign language interpretation has been used to address this issue, the lack of a more intuitive means of conveying information has made it impossible to effectively convey aspects such as a dancer's movements and the texture of the dance. Furthermore, processing information through motion capture has been limited by post-processing techniques and complex software, requiring specialized environments. Furthermore, the interfaces are often difficult for those with hearing or visual impairments to use, making them difficult to use.

[0003] One embodiment of the present invention aims to reduce the gap in dance acquisition and appreciation between hearing- and visually-impaired people and the general public by converting audiovisual information into tactile information and delivering it to the user.

[0004] One embodiment of the present invention aims to secure a wider creative space than a motion capture method through a strain sensor.

[0005] The present invention relates to an alternative sensory transmission system, and provides an alternative sensory transmission system comprising: a motion detection unit that detects the movements of a dancer; a conversion unit that converts information detected by the motion detection unit into tactile information; and a haptic unit that implements the tactile information converted by the conversion unit into a tactile signal to a receiver.

[0006] In addition, the motion detection device may include a strain sensor that is worn on the dancer's body and applied to a plurality of predetermined locations on the dancer's body.

[0007] Additionally, the conversion unit can convert the speed and displacement of the motion detected by the strain sensor into one or more of the haptic intensity, duration, and response pattern.

[0008] In addition, the haptic unit may include a first mode in which the haptic unit transmits a tactile signal when the recipient takes a motion detected by the motion detection unit; and a second mode in which the haptic unit transmits a sound source converted into a tactile signal to the user.

[0009] In addition, the device may further include a storage unit that stores tactile information converted into a dancer's dance pattern by the conversion unit as a dance score.

[0010] In addition, it further includes a motion recognition unit that recognizes the movements of a dancer, and the genre of the dance can be distinguished and stored in the storage unit through motion recognition information.

[0011] Additionally, the haptic unit can transmit signals to at least the same body part as the motion detection unit.

[0012] Additionally, the strain sensor can be electrically connected to a conductive electrode woven into the fiber.

[0013] Additionally, strain sensors are placed at positions corresponding to the dancer's joints, and the tension of multiple strain sensors adjacent to each other at the same time can be machine-learned to infer the dancer's posture.

[0014] Additionally, the dance can be created by the user accessing it by generating one or more different factors among the sound source, body part, dancer's movement, size and speed of movement.

[0015] Additionally, the motion detection device may include one or more of a magnetometer and an optical sensor that are worn on the dancer's body and applied to a plurality of predetermined locations on the dancer's body.

[0016] According to one embodiment of the present invention, an alternative sensory transmission system can be provided that reduces the gap in dance acquisition and appreciation between hearing- and visually-impaired people and the general public by converting audiovisual information into tactile information and transmitting it to the user.

[0017] According to one embodiment of the present invention, an alternative sensory transmission system that secures a wider creative space than a motion capture method can be provided through a strain sensor.

[0018] FIG. 1 is a schematic diagram showing an alternative sensory transmission system according to one embodiment of the present invention.

[0019] FIG. 2 is a diagram showing a path through which data is transmitted in an alternative sensory transmission system according to one embodiment of the present invention.

[0020] FIG. 3 is a drawing showing the configuration of a haptic unit according to one embodiment of the present invention.

[0021] FIG. 4 is a schematic diagram showing a first mode for accessing a storage unit and uploading a new creation based on an existing music score according to one embodiment of the present invention, and FIG. 5 is a schematic diagram showing a second mode for transmitting a music score stored in a storage unit along with a sound source to a recipient according to one embodiment of the present invention.

[0022] Figure 6 is an example of a graph showing information detected from a strain sensor according to one embodiment of the present invention.

[0023] FIG. 7 is a photograph of an actual configuration in which motion detection units are attached to a vest and an arm to detect motions such as waves and bounces according to one embodiment of the present invention.

[0024] Figure 8 is a drawing explaining the steps for recognizing the operation of Figure 7.

[0025] Hereinafter, specific embodiments of the present invention will be described with reference to the drawings. However, these are merely examples and the present invention is not limited thereto.

[0026] In describing the present invention, detailed descriptions of known technologies related to the present invention will be omitted if they are deemed to unnecessarily obscure the gist of the invention. Furthermore, the terms described below are defined based on their functions within the present invention and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the overall content of this specification.

[0027] The technical idea of ​​the present invention is determined by the claims, and the following examples are merely a means of efficiently explaining the technical idea of ​​the present invention to a person having ordinary skill in the technical field to which the present invention belongs.

[0028]

[0029] FIG. 1 is a schematic diagram showing an alternative sensory transmission system according to one embodiment of the present invention.

[0030] Referring to FIG. 1, the alternative sensory transmission system of the present invention may include a motion detection unit (100) that detects the motion of a dancer (10), a conversion unit (200) that converts information detected from the motion detection unit (100) into tactile information, and a haptic unit (300) that implements the tactile information converted by the conversion unit (200) as a tactile signal (ST) to a receiver (20).

[0031] The above motion detection unit (100) may employ at least one of a strain sensor (110), a magnetometer, and an optical sensor, and a plurality of sensors are positioned on a specific body of the dancer (10) to detect the distance between the sensors or the amount of change in tensile or compressive energy, so that the movements of the dancer (10) can be recorded as data. In one embodiment of the present invention, the strain sensor (110) applied to the motion detection unit (100) will be described below. Of course, the magnetic moment sensor (magnetometer) and the optical sensor that detect the amount of energy or the amount of distance (displacement) may also be applied, and for this purpose, it is preferable to place them adjacent to a specific part (for example, a joint) of the body of the dancer (10).

[0032] The above motion detection unit (100) may include a strain sensor (110) that is worn on the body of the dancer (10) and applied to a plurality of predetermined locations on the body of the dancer (10).

[0033] The above strain sensor (110) can generate tensile force according to the movement of the dancer (10) and transmit the detected tensile force to the conversion unit (200). Furthermore, the device can further include a motion recognition unit (170) capable of motion recognition (motion capture) of the dancer (10) using a camera. The motion recognition unit (170) can recognize the movements of the dancer (10) using a motion capture method and transmit the recognition to the conversion unit (200) so that the genre of the dance can be distinguished.

[0034] The above dancer (10) can detect the movement during the dance through the strain sensor (110), and the detected tension can be converted into tactile information by the operation unit (220) included in the conversion unit (200). The converted tactile information can be transmitted to the receiver (20). At this time, the haptic unit (300) in contact with the receiver (20) can transmit a signal to at least the same body part as the motion detection unit (100). Preferably, the motion detection unit (100) and the haptic unit (300) have a wearable form and can maintain contact with the body by being worn. Therefore, the tension detected for each body part through the strain sensor (110) can be transmitted as tactile information to the receiver (20) through the conversion unit (200).

[0035] That is, since the tension detected for a predetermined body position causes the haptic performing unit (320) located at the corresponding position to react, tactile information can be transmitted immediately through signal conversion. This eliminates the need for post-processing such as forming feature points for each body part in the past, performing modeling work through a computer, and then calculating movement, and the calculation unit (220) intervenes to convert the tension detected by the strain sensor (110) into tactile information. Of course, this is different tactile information from the information transmitted after modeling work, so the present invention has a clear difference in mechanism from the prior art.

[0036] Meanwhile, a position sensor (150) that detects position information may be provided to identify the dancer's (10) movement and position when the dance unfolds within an arbitrary area (TA). The position information of the dancer (10) may also be transmitted to a conversion unit (200) and selectively transmitted to a receiver (20).

[0037] FIG. 2 is a diagram showing a path through which data is transmitted in an alternative sensory transmission system according to one embodiment of the present invention.

[0038] Referring to FIG. 2, the device may further include a storage unit (225) that stores tactile information converted into a dance pattern of a dancer (10) by a conversion unit (200) as a dance score. The dance score may be formed based on at least the information detected by the strain sensor (110) among the information detected by the motion recognition unit (170), the position sensor (150), and the strain sensor (110) of the motion detector.

[0039] As described above, information detected from the motion recognition unit (170), the position sensor (150), and the strain sensor (110) of the motion detector can be transmitted to the conversion unit (200). Specifically, the detected information can be transmitted to the receiving unit (210) of the conversion unit (200). The information received by the receiving unit (210) can be operated by the operating unit (220). The operating unit (220) can form a motion signal and store it in the storage unit (225) while simultaneously converting it into tactile information. The motion signal can be stored in the storage unit (225) like a database, and the converted tactile information can be transmitted to the haptic unit (300) via the transmitting unit (230).

[0040] The above haptic unit (300) can receive transmitted tactile information through the communication unit (310). The received tactile information can be transmitted to the haptic execution unit (320) and implemented through a predetermined haptic response. The haptic response can be implemented through a response maintenance time, response intensity, a predetermined pattern, etc., thereby distinguishing information about movement.

[0041] Meanwhile, the motion recognition unit (170) can store motion recognition information in the storage unit (225) as a factor for distinguishing dance genres, and can be exposed to be selected by genre through user access (A1). This will be described in detail with reference to FIG. 5 below.

[0042] The signal transmitted to the above communication unit (310) can transmit a tactile signal (ST) to the haptic execution unit (320), and each tactile signal (ST) can include location information. For example, the tactile information processed from the information detected by the strain sensor (110) located on the left shoulder of the dancer (10) in the motion detection unit (100) includes the location information, and thus, when transmitted to the receiver (20) through the communication unit (310), a tactile signal (ST) can be transmitted to the left shoulder.

[0043] The above communication unit (310) can be electrically connected to transmit the received information to multiple haptic execution units (320). This will be described in detail with reference to FIG. 3 below.

[0044]

[0045] FIG. 3 is a drawing showing the configuration of a haptic unit (300) according to one embodiment of the present invention.

[0046] Referring to FIG. 3, the haptic performing unit (320) can be electrically connected to a conductive electrode woven with a fiber yarn (302). Specifically, as described above, the haptic performing unit (300) formed to be wearable by the receiver (20) may be made of a fabric, and may be woven to include electrode yarn (301), which is a thin conductive wire, to be used as an electrode, although it is made of a fiber yarn (302). Accordingly, the haptic performing unit (300) can react to a signal having matching position information among the tactile signals (ST) received by the communication unit (310). The position information may be a type of code. Accordingly, it is preferable that the strain lines of the motion detection unit (100) and the haptic performing unit (320) of the haptic performing unit (300) be arranged in the same number and at the same position.

[0047] Even if it is a microcurrent, the surface that comes into contact with the human body may not be directly exposed and a fiber (302) may be added. Of course, the fiber (302) formed on the surface where the haptic part (300) is exposed, i.e., the surface facing the human body, may be composed of a material of insulating material woven into fine lines.

[0048]

[0049] FIG. 4 is a schematic diagram showing a first mode of uploading a new creation based on an existing song by accessing (A1) a storage unit (225) according to one embodiment of the present invention, and FIG. 5 is a schematic diagram showing a second mode of transmitting a song stored in a storage unit (225) along with a sound source (S) to a recipient (20) according to one embodiment of the present invention.

[0050] Referring to FIG. 4, in the first mode, the haptic unit (300) transmits a tactile signal (ST) when the receiver (20) takes a motion detected by the motion detection unit (100). That is, when the tactile signal (ST) is sequentially transmitted to the receiver (20) through the communication unit (310) and the haptic execution unit (320) according to the previously stored dance score through the first upload (U1) to the storage unit (225), the receiver (20) may learn the dance following the guide of the dance score, but may also take a motion that deviates from the guide based on the dance score. A new dance score including the created dance can be formed through such motions. The new dance score can be newly stored in the storage unit (225) through the second upload (U2). In this way, the user can freely access (A1) to learn the previously stored dance score or create the dance score with his or her own motion. Through these activities, the storage unit (225) can accumulate a database.

[0051] Meanwhile, referring to FIG. 5, a second mode may be included for transmitting a sound source (S) converted into a tactile signal (ST) to the user. The second mode allows the user to enjoy music that includes sound source (S) information and a tactile signal (ST) stored in the storage unit (225). At this time, the operation unit (220) separately forms tactile information according to the sound source (S), so that the sound source (S) and the previously stored tactile signal (ST) can be felt in a composite manner.

[0052]

[0053] FIG. 6 is an example of a graph showing information detected from a strain sensor (110) according to one embodiment of the present invention.

[0054] Referring to FIG. 6, the conversion unit (200) can convert the speed and displacement of the motion detected by the strain sensor (110) into one or more of the intensity, duration, and response pattern of the haptic. As described above, the receiver (20) can have the haptic performing unit (320) placed at the same location as the strain sensor (110) placed on the body part of the dancer (10) in order to recognize various motions of the dancer (10). Furthermore, the haptic response can be implemented differently based on information based on the tension detected by the strain sensor (110) in order to detect the degree and speed of the movement of a specific body part.

[0055] As illustrated, the vertical axis A may represent the angle (displacement) of the joint, the vertical axis D may represent the tensile distance (tension), and the horizontal axis T may represent time. Through the displacement and tension occurring over time, the calculation unit (220) may analyze which motion occurred at which point and form tactile signals (ST) that are the same or different.

[0056] As an example for performing such detection, the strain sensor (110) may be placed at a position corresponding to the joint of the dancer (10). At the same time, the tension of a plurality of strain sensors (110) adjacent to each other may be machine-learned to infer the posture of the dancer (10). For example, during the playback of a sound source (S), information on the rotation of the wrist, elbow, and shoulder at a specific time may be compared and accumulated through a plurality of dancers (10), and movements corresponding to the sound source (S) may be learned, thereby forming a dance score.

[0057] The dance formed in this way can also be created by a user who accesses (A1) the storage unit (225) as described through the second mode, by generating one or more different factors among the sound source (S), body part, movement line of the dancer (10), size and speed of the movement.

[0058]

[0059] While representative embodiments of the present invention have been described in detail above, those skilled in the art will appreciate that various modifications to the above-described embodiments are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims set forth below but also by equivalents thereof.

[0060] FIG. 7 is a photograph of an actual configuration in which motion detection units are attached to a vest and an arm to detect motions such as waves and bounces according to one embodiment of the present invention.

[0061] Referring to Fig. 7, it can be seen that an arm wave (top of Fig. 7) can be recognized by the motion detection unit attached to the arm, a bounce (middle of Fig. 7) by the motion detection unit of the chest vest, and a swing (hop and head, bottom of Fig. 7) by the movement of the motion detection units of the arm and vest.

[0062] Figure 8 is a drawing explaining the steps for recognizing the operation of Figure 7.

[0063] Referring to Figure 8, sensor-based motion is first detected. Machine learning is then performed based on dance pattern recognition to digitize the data. A symbolic haptic pattern is then generated, and the information detected by the motion detection unit is converted into tactile information.

[0064] The present invention relates to an alternative sensory transmission system and has industrial applicability.

Claims

1. A motion detection unit that detects the dancer's movements; A conversion unit that converts information detected from the above motion detection unit into tactile information; and An alternative sensory transmission system, comprising a haptic unit that converts the tactile information into a tactile signal to a receiver.

2. In claim 1, The motion detection device is An alternative sensory transmission system comprising a strain sensor worn on the dancer's body and applied to a plurality of predetermined locations on the dancer's body.

3. In claim 2, The above conversion part, An alternative sensory transmission system that converts the speed and displacement of the motion detected by the above strain sensor into one or more of the intensity, duration, and response pattern of haptics.

4. In claim 1, The haptic part is, An alternative sensory transmission system comprising: a first mode in which the haptic unit transmits the tactile signal when the receiver performs a motion detected by the motion detection unit; and a second mode in which the sound source converted into the tactile signal is transmitted to the user.

5. In claim 1, An alternative sensory transmission system further comprising a storage unit that stores tactile information converted into a dance pattern of the dancer by the conversion unit as a dance score.

6. In claim 5, Further comprising a motion recognition unit that recognizes the movements of the dancer, An alternative sensory transmission system that stores dance genres in the storage unit by distinguishing them through the above motion recognition information.

7. In claim 1, An alternative sensory transmission system in which the above haptic unit transmits a signal to at least the same body part as the above motion detection unit.

8. In claim 2, The above strain sensor is an alternative sensory transmission system that is electrically connected to a conductive electrode woven from fiber yarn.

9. In claim 2, The strain sensor is positioned at a position corresponding to the joint of the dancer, and the tension of a plurality of adjacent strain sensors at the same time is machine-learned to infer the posture of the dancer.

10. In claim 5, An alternative sensory transmission system in which a created dance score can be generated and stored in the storage unit when an accessed user generates one or more different factors among a sound source, a body part, the dancer's movement line, and the size and speed of the movement.

11. In claim 1, The above motion detection unit, An alternative sensory transmission system, which is worn on the dancer's body and includes at least one of a magnetometer and an optical sensor applied to a plurality of predetermined locations on the dancer's body.

12. In paragraph 11, The above motion detection unit, An alternative sensory transmission system characterized by being attached to a vest worn on the dancer's body, or a band attached to an arm or leg.

Images