Tactile stimulus providing system

Abstract

To provide a tactile stimulation providing system capable of providing a tactile stimulation suitable for a situation by using an external program.SOLUTION: The tactile stimulation providing system includes: a tactile stimulation pattern generating apparatus that generates first actuator strength values corresponding to representative values of frequency bands of first audio data and second actuator strength values corresponding to representative values of frequency bands of second audio data; and a tactile stimulation providing apparatus that drives actuators based on the first actuator strength values and the second actuator strength values described above. An actuator corresponding to first position information corresponding to a first direction is driven based on only at least one of the first actuator strength values. An actuator corresponding to second position information corresponding to a second direction is driven based on only at least one of the second actuator strength values.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a tactile stimulation delivery system. [Background technology]

[0002] The tactile stimulation providing device includes a plurality of actuators and can provide a tactile stimulation to a user by selectively vibrating the plurality of actuators. When enjoying audiovisual-based content such as games and videos using a mobile device, console, computer, virtual reality terminal, etc., if the tactile stimulation is transmitted to the user, the user can enjoy the content and feel immersive.

[0003] However, for example, if the tactile stimulation function is not built into the game from the time of development of the game program, a new method of providing tactile stimulation suitable for each game situation must be researched using an external program. Summary of the Invention [Problem to be solved by the invention]

[0004] The technical problem to be solved is to provide a tactile stimulation providing system that can provide tactile stimulation appropriate to a situation by utilizing an external program. [Means for solving the problem]

[0005] A tactile stimulus providing system according to an embodiment of the present invention includes a tactile stimulus pattern generating device that generates n first actuator strength values ​​corresponding to a representative value of a frequency band of first audio data and m second actuator strength values ​​corresponding to a representative value of a frequency band of second audio data, and a tactile stimulus providing device that includes p actuators and drives the p actuators based on the n first actuator strength values ​​and the m second actuator strength values, where n, m, and p are integers greater than 1 and p is less than a sum of n and m, the first audio data includes information about a first direction, the second audio data includes information about a second direction different from the first direction, an actuator corresponding to first position information corresponding to the first direction is driven based on at least one of the first actuator strength values, and an actuator corresponding to second position information corresponding to the second direction is driven based on at least one of the second actuator strength values.

[0006] The first audio data may correspond to data of a first channel of stereo audio data, and the second audio data may correspond to data of a second channel of the stereo audio data.

[0007] The first audio data may correspond to first channel data of surround audio data having three or more channels, and the second audio data may correspond to second channel data of the surround audio data.

[0008] At least one of the p actuators may be driven based on at least one of the n first actuator strength values ​​and at least one of the m second actuator strength values.

[0009] At least one of the p actuators may be driven based on at least two of the n first actuator strength values.

[0010] At least two of the frequency bands may overlap at least partially.

[0011] The tactile stimulation pattern generating device may include a data conversion unit for converting the first audio data and the second audio data in the time domain into first converted data and second converted data in the frequency domain, respectively.

[0012] The tactile stimulation pattern generating device may further include a band representative value calculation unit that calculates the representative value of the frequency band for each of the first converted data and the second converted data, and the representative value may be based on valid data that is greater than a minimum threshold value and less than a maximum threshold value for each of the frequency bands.

[0013] The tactile stimulation pattern generation device may further include an actuator strength calculation unit that calculates an actuator strength value corresponding to the representative value.

[0014] The actuator strength calculation section can calculate the actuator strength value so that the smaller the representative value is, the smaller the actuator strength value is, and the larger the representative value is, the smaller the actuator strength value is.

[0015] The actuator strength calculation section may have a relationship between the representative value and the actuator strength value that includes a discontinuous function.

[0016] A first minimum threshold corresponding to a first one of the frequency bands may be different from a second minimum threshold corresponding to a second one of the frequency bands, and a first maximum threshold corresponding to the first frequency band may be different from a second maximum threshold corresponding to the second frequency band.

[0017] The tactile stimulation pattern generating apparatus may further include an actuator strength attenuation unit that gradually attenuates the actuator strength value when the actuator strength value is maintained for a certain period of time or more.

[0018] The tactile stimulation pattern generating device may further include an actuator strength merging unit that generates one actuator strength value by merging two or more actuator strength values ​​mapped to one actuator.

[0019] The tactile stimulation pattern generating device may further include an automatic setting unit that calculates an effective range of the representative value for each of the frequency bands based on a frequency distribution of the representative value over time, sets the minimum threshold value to correspond to the minimum value of the effective range, and sets the maximum threshold value to correspond to the maximum value of the effective range.

[0020] A computer-readable recording medium according to one embodiment of the present invention is a computer-readable recording medium having a computer-executable program recorded thereon, the program creating a user interface, the user interface including a channel setting area including a frequency band of at least one audio channel among two or more audio channels, and a local setting area including a local setting value of a frequency band selected from the frequency bands, the local setting value including a virtual actuator selected from a plurality of virtual actuators corresponding to the selected frequency band.

[0021] The local settings may include a maximum threshold that is an upper limit of valid data for the selected frequency band, and a minimum threshold that is a lower limit of the valid data.

[0022] The user interface may further include a global setting area including global setting values ​​that are commonly applied to the frequency bands.

[0023] The user interface may further include a mode selection area in which one of a stereo audio mode having two audio channels and a surround audio mode having three or more audio channels is selectable.

[0024] The user interface may further include a simulation area for visualizing the drive strength of the virtual actuator in response to input audio data. Effect of the Invention

[0025] The tactile stimulus providing system according to the present invention can provide a tactile stimulus appropriate for a situation by using an external program. [Brief description of the drawings]

[0026] [Figure 1] 1 is a diagram illustrating a tactile stimulus providing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a view for explaining an actuator case according to an embodiment of the present invention from one perspective. [Diagram 3] 3 is a diagram for explaining the actuator case in FIG. 2 from another perspective. FIG. [Figure 4] 1 is a front view of an arm cover-shaped tactile stimulation providing device according to one embodiment of the present invention. [Diagram 5] FIG. 5 is a rear view of the tactile stimulation providing device of FIG. 4. [Figure 6] 5 is a diagram showing the tactile stimulation providing device of FIG. 4 being worn by a user. FIG. [Figure 7] 5 is a diagram showing a state in which the tactile stimulus providing device of FIG. 4 is worn by a user. [Figure 8] FIG. 13 is a view for explaining an actuator case according to another embodiment of the present invention from one perspective. [Figure 9] 9 is a diagram for explaining the actuator case in FIG. 8 from a different perspective. [Figure 10] 1 is a front view of a vest-shaped tactile stimulation providing device according to an embodiment of the present invention; [Figure 11] FIG. 11 is a rear view of the tactile stimulation providing device of FIG. [Figure 12] 12 is a diagram for explaining the tactile stimulus providing device of FIG. 11 with the front panel opened. FIG. [Figure 13] FIG. 2 is a diagram for explaining a user interface according to an embodiment of the present invention. [Figure 14] FIG. 14 is a diagram for explaining a simulation area of ​​the user interface of FIG. 13. [Figure 15] FIG. 14 is a diagram for explaining a simulation area of ​​the user interface of FIG. 13. [Figure 16] 1 is a diagram for explaining a tactile stimulation pattern generating device according to an embodiment of the present invention; [Figure 17] FIG. 17 is a diagram for explaining a data conversion unit of the tactile stimulation pattern generating device of FIG. 16. [Figure 18] 17 is a diagram for explaining a band representative value calculation unit of the tactile stimulation pattern generation device of FIG. 16. FIG. [Figure 19] 17 is a diagram for explaining an actuator strength calculation unit of the tactile stimulation pattern generation device of FIG. 16. FIG. [Figure 20] 17 is a diagram for explaining an actuator strength calculation unit of the tactile stimulation pattern generation device of FIG. 16. FIG. [Figure 21] 17 is a diagram for explaining an actuator strength calculation unit of the tactile stimulation pattern generation device of FIG. 16. FIG. [Figure 22] 17 is a diagram for explaining an actuator strength calculation unit of the tactile stimulation pattern generation device of FIG. 16. FIG. [Figure 23] 17 is a diagram for explaining an actuator intensity merging unit of the tactile stimulation pattern generating device of FIG. 16. FIG. [Figure 24] 1 is a diagram for explaining an exemplary relationship between a tactile stimulation pattern generating device and a tactile stimulation providing device. [Diagram 25] FIG. 13 is a diagram illustrating a tactile stimulation pattern generating device according to another embodiment of the present invention. [Figure 26]FIG. 11 is a diagram for explaining a user interface according to another embodiment of the present invention. [Figure 27] FIG. 13 is a diagram for explaining a path effect which is a unit effect. [Figure 28] FIG. 11 is a diagram for explaining a dot effect which is a unit effect. [Figure 29] FIG. 2 is a diagram for explaining the arrangement of a tactile stimulation frame according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention may, however, be embodied in various different forms and should not be construed as being limited to the embodiments set forth herein.

[0028] In order to clearly explain the present invention, parts that are not relevant to the description will be omitted, and the same or similar components will be given the same reference numerals throughout the specification. Therefore, the above reference numerals can be used in other drawings.

[0029] In addition, for convenience of explanation, the size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention is not necessarily limited to the contents shown in the drawings. In the drawings, thicknesses may be exaggerated to clearly show multiple layers and regions.

[0030] Referring to FIG. 1, a tactile stimulus providing system 8 according to an embodiment of the present invention includes a tactile stimulus pattern generating device 9 and a tactile stimulus providing device 10.

[0031] The tactile stimulation pattern generating device 9 can generate n first actuator strength values ​​corresponding to representative values ​​of a frequency band of the first audio data and m second actuator strength values ​​corresponding to representative values ​​of a frequency band of the second audio data. The tactile stimulation pattern generating device 9 can provide a tactile stimulation pattern TSP based on the first actuator strength values ​​and the second actuator strength values ​​to the tactile stimulation providing device 10.

[0032] According to one embodiment, the first audio data may correspond to data of a first channel of stereo audio data, and the second audio data may correspond to data of a second channel of stereo audio data.

[0033] According to yet another embodiment, the first audio data may correspond to data of a first channel of surround audio data having three or more channels, and the second audio data may correspond to data of a second channel of the surround audio data. The surround audio data may correspond to existing 5.1 channel surround audio data, 7.1 channel surround audio data, etc.

[0034] According to the above embodiment, since the representative value of a specific frequency band in the frequency domain is used instead of using the original audio data in the time domain as it is, the amount of data for each audio channel included in the tactile stimulation pattern TSP is reduced. That is, assuming that the data transmission rate from the tactile stimulation pattern generating device 9 to the tactile stimulation providing device 10 is fixed, it is possible to transmit a tactile stimulation pattern TSP including data for more audio channels. Therefore, the tactile stimulation providing system 8 according to the embodiment of the present invention is compatible with various audio modes such as a two-channel stereo audio mode, a 5.1-channel surround audio mode, and a 7.1-channel surround audio mode.

[0035] The tactile stimulus providing device 10 includes p actuators, and is capable of driving the p actuators based on n first actuator strength values ​​and m second actuator strength values.

[0036] In this case, n, m, and p may be integers greater than 1. p may be less than the sum of n and m. That is, when the actuator strength values ​​are mapped to the actuators, the number of actuator strength values ​​may be greater than the number of actuators. In such a case, at least one actuator may be driven based on at least two actuator strength values.

[0037] For example, at least one of the p actuators may be driven based on at least one of the n first actuator strength values ​​and at least one of the m second actuator strength values, i.e., the drive strength of the at least one actuator may be determined based on both the first audio data and the second audio data.

[0038] Also, for example, at least one of the p actuators may be driven based on at least two of the n first actuator strength values, i.e., the drive strength of the at least one actuator may be determined based not only on the first audio data but also on representative values ​​of at least two frequency bands.

[0039] According to the above-mentioned embodiment, it is possible to emphasize the directionality of the sound depending on the position of the actuator, or to emphasize the acoustic characteristics of the actuator. The effect of emphasizing the directionality depending on the position of the actuator will be described later with reference to Fig. 24. The effect of emphasizing the acoustic characteristics of the actuator will be described later with reference to Fig. 18.

[0040] In the embodiment of the present invention, the actuator strength value or driving strength does not mean only the magnitude of one vibration of the actuator, but may be embodied in various ways by appropriately adjusting the magnitude of one vibration, driving time, driving frequency, driving pattern, etc.

[0041] According to one embodiment, the tactile stimulation pattern generator 9 is capable of generating the tactile stimulation pattern TSP via a user interface that includes a number of virtual actuators onto which the positions of the actuators are mapped.

[0042] The tactile stimulation pattern generating device 9 may be a computer including at least one processor and a memory for information processing. The tactile stimulation pattern generating device 9 can execute a desired function by reading a recording medium in which data or a program is recorded.

[0043] The recording medium includes any kind of recording device that can store data or programs that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, hard disk, external hard disk, SSD, USB storage device, DVD, Blu-ray disk, etc. In addition, the computer-readable recording medium may be a combination of multiple devices, or may be distributed among computer systems connected by a network. The recording medium may be a non-transitory computer readable medium. The non-transitory computer readable medium means a medium that stores data or programs semi-permanently and is readable by a computer, as opposed to a medium that stores data or programs for a short period of time, such as a register, cache, or memory.

[0044] The user interface may be realized by a display device that displays visual information to a user and an input device that receives user input. The display device may include a monitor, a TV, a projector, a mobile display, etc. The input device may include a mouse, a keyboard, a touch pad, a microphone, a vision sensor, etc. The display device and the input device are not necessarily separate, and for example, the display device and the input device may be realized as an integrated device such as a touch display device. The user interface may be provided by a tactile stimulation pattern generating device 9.

[0045] The actuators refer to a plurality of tactile stimulus providing modules physically present in the tactile stimulus providing device 10. The actuators may be vibration motors, which are an electric actuator. In the following description, the actuators are vibration motors, but the actuators may be pneumatic, hydraulic, EAP (electroactive polymer), or piezoelectric actuators. The actuators may be electrical stimulation or temperature transmission devices. Thus, one or more types of actuators may be applied to the present invention.

[0046] The user interface includes a plurality of virtual actuators to which the positions of the plurality of actuators are mapped. The user interface can be graphically displayed so that the user can intuitively recognize the physical arrangement of the plurality of actuators included in the tactile stimulus providing device 10. For example, if the tactile stimulus providing device 10 includes a plurality of actuators in 5 rows and 5 columns, the user interface can include a plurality of virtual actuators in 5 rows and 5 columns. As another example, if the tactile stimulus providing device 10 includes a plurality of actuators in 1 row and 5 columns, the user interface can include a plurality of virtual actuators in 1 row and 5 columns.

[0047] In other embodiments, the actuators of the tactile stimulus providing device 10 and the virtual actuators of the user interface may not correspond one-to-one. For example, a tactile stimulus pattern generated for a plurality of virtual actuators of 1 row and 5 columns in the tactile stimulus pattern generating device 9 may correspond to one row of a plurality of actuators of 5 rows and 5 columns in the tactile stimulus providing device 10. Therefore, a method of matching a plurality of actuators with a plurality of virtual actuators may vary depending on the embodiment.

[0048] The tactile stimulus pattern TSP generated by the tactile stimulus pattern generating device 9 may be data or a program provided at the input of the tactile stimulus providing device 10 .

[0049] The tactile stimulus providing device 10 includes a plurality of actuators, and drives the plurality of actuators in accordance with the generated tactile stimulus pattern TSP. The drive time, drive frequency, drive strength, drive interval, etc. of the plurality of actuators are adjusted in accordance with the tactile stimulus pattern, so that the user of the tactile stimulus providing device 10 can feel various tactile stimuli.

[0050] FIG. 2 is a view for explaining an actuator case according to an embodiment of the present invention from one perspective, and FIG. 3 is a view for explaining the actuator case of FIG. 2 from another perspective.

[0051] 2 and 3, an actuator case 1 according to an embodiment of the present invention includes a bottom case bc1, a cover case cc1, and a top case tc1.

[0052] The positional relationship between the bottom case bc1, the cover case cc1, and the top case tc1 may be determined based on the baseband. The baseband may be located between the cover case cc1 and the top case tc1. For the relative positional relationship between the baseband and the actuator case 1, further refer to the description of FIGS. 3 and 4.

[0053] The actuator ac1 is located on one side of the baseband and includes a driver mb1 and an eccentric mass em1. The driver mb1 and the eccentric mass em1 may be rotatably connected via a shaft. The actuator ac1 may be an eccentric rotation motor (ERM motor). The actuator ac1 may generate vibrations by having the eccentric mass em1 rotated by the driver mb1 act as an unbalanced vibrator. The actuator ac1 may be a cylindrical vibration motor with a cylinder diameter of about 6 mm and a total height of about 17 mm. In a coin-type motor, the vibrator rotates horizontally to generate horizontal vibrations, but the actuator ac1 of this embodiment can generate vertical vibrations.

[0054] The bottom case bc1 is located on one side of the baseband and includes a first housing portion tha1 that houses the driver mb1 and a second housing portion thb1 that houses the eccentric mass em1. That is, the bottom case bc1 can house all or a part of the actuator ac1 except for the wiring. Depending on the embodiment, when the bottom case bc1 houses only a part of the actuator ac1, the remaining part of the actuator ac1 may be housed by a cover case cc1 described later.

[0055] A cover case cc1 is located on one side of the baseband and is coupled to the bottom case bc1 so as to bring the driver mb1 into close contact with the first receiving portion tha1.

[0056] 2 and 3, the cover case cc1 may be coupled to the bottom case bc1 while covering the actuator ac1. The male fastening portion cm1 of the cover case cc1 may be fitted into the female fastening portion bsf1 of the bottom case bc1 to firmly fix the cover case cc1 to the bottom case bc1. In particular, a first receiving portion tha11 and a receiving portion cha1 may be formed so that the driving portion mb1 of the actuator ac1 is firmly fixed. The male fastening portion cm1 of the cover case cc1 may protrude parallel to a surface where the cover case cc1 abuts against the baseband. In this embodiment, there are four pairs of male fastening portions cm1 and female fastening portions bsf1, but the number may vary depending on the embodiment.

[0057] The cover case cc1 is coupled to the bottom case bc1, isolating the eccentric mass em1 from one side of the baseband, and can form a sufficient rotation space for the eccentric mass em1 together with the second housing part thb1. That is, the housing part chb1 of the cover case cc1 is matched with the second housing part thb1, and can form a sufficient rotation space that does not impede the rotation of the eccentric mass em1. The housing part chb1 of the cover case cc1 also serves to isolate the eccentric mass em1 from one side of the baseband. Therefore, even if the baseband is made of a flexible material such as cloth or rubber, the eccentric mass em1 can rotate reliably without the risk of hitting the baseband.

[0058] A top case tc1 is located on the other side of the baseband and fixes the bottom case bc1 to the baseband.

[0059] The male fastening portion tm1 of the top case tc1 penetrates the baseband and is coupled to the female fastening portion bf1 of the bottom case, thereby fixing the actuator ac1 and the actuator case 1 to the baseband. In this case, the baseband may include an opening shaped to allow the male fastening portion tm1 to pass through. Depending on the embodiment, the area of ​​the top case tc1 that is in close contact with the other side of the baseband may be smaller than the area of ​​the bottom case bc1 that is in close contact with one side of the baseband. In this case, since the space in which the male fastening portion tm1 can be located is relatively insufficient, in this embodiment, the male fastening portion tm1 and the female fastening portion bf1 are formed in two pairs. According to this embodiment, if the area of ​​the top case tc1 is smaller than the area of ​​the bottom case bc1, even if multiple actuators are arranged closely together, the interval between the multiple top cases where the vibrations of each are concentrated becomes sufficient, and there is an advantage that the resolution of the vibration felt by the user is improved. In addition, in this embodiment, one side of the top case tc1 is configured in a circular shape, so that the user does not feel any corners and can feel more comfortable.

[0060] The bottom case bc1 may include a protrusion bm1, and the top case tc1 may include a recess tf1. The protrusion bm1 of the bottom case bc1 presses against the base band and fits into the recess tf1, so that the actuator ac1 and the actuator case 1 can be fixed to the base band more firmly. In this embodiment, since the area of ​​the top case tc1 is relatively small, four pairs of protrusions bm1 and recesses tf1 are formed accordingly.

[0061] FIG. 4 is a front view of an arm cover-type tactile stimulus providing device according to one embodiment of the present invention, and FIG. 5 is a rear view of the tactile stimulus providing device of FIG.

[0062] 4 and 5, an arm cover-shaped tactile stimulation providing device 10a according to an embodiment of the present invention may include a base band 100, a plurality of actuators, a plurality of bottom cases, a plurality of cover cases, and a plurality of top cases 500c. Depending on the embodiment, the tactile stimulation providing device 10 may selectively further include an auxiliary band 200, a controller case 300, a ring 400, a controller, and a signal transmission member.

[0063] A plurality of top cases 500c may be located on the rear surface of the baseband 100. A plurality of bottom cases and cover cases may be located on the front surface of the baseband 100. The plurality of bottom cases and cover cases may be covered by a protective cover CV and a detachable member 110, 120. A plurality of actuators may be located between the corresponding bottom cases and cover cases. Thus, the baseband 100 may support a plurality of actuators.

[0064] The base band 100 may be made of a flexible material such as a cloth material, a mesh material, a rubber material, or neoprene. Since users have various shapes and sizes even in the same body part, when a flexible material is used for the base band 100, it can provide a comfortable fit to the user. The base band 100 may be made of a single layer, but in a preferred embodiment, it may be made of multiple layers. The multiple layers may be made of different materials. For example, the back of the base band 100 may be made of a layer of mesh material, and the front of the base band 100 may be made of a layer of neoprene material. In this case, multiple actuators may be located on the back of the base band 100 made of a mesh material, and the mesh material can prevent the vibrations of the multiple actuators from spreading and can prevent the movement of the multiple actuators. In addition, the front of the base band 100 made of a neoprene material can prevent the vibrations generated by the multiple actuators from reaching the controller case 300.

[0065] The detachable member 110 may be located on the front surface of the baseband 100. Also, the detachable member 120 may be located at the other end of the front surface of the baseband 100, spaced apart from the detachable member 110, and may be detachable from the detachable member 110.

[0066] The detachable member 110 may be an A type detachable member, and the detachable member 120 may be a B type detachable member.

[0067] In this embodiment, the detachable member may be any one of type A and type B. Type A and type B are detachable from each other. However, type A and type A may not be detachable from each other. Similarly, type B and type B may not be detachable from each other. Any of these types of detachable members may be used as the detachable member of this embodiment. The detachable member may include, for example, Velcro, a magnet, an electromagnet, a snap button, a buckle, etc. In the drawings of the following embodiment, it is assumed that the detachable member is Velcro.

[0068] The upper and lower edges of the baseband 100 may be streamlined. The expanded shape of the baseband 100 is streamlined like a development of a truncated cone, so that the baseband 100 can be more closely fitted to the user's arm when worn.

[0069] The auxiliary band 200 may include a detachable member 110 and a detachable type B detachable member (not shown) on the rear surface, and one end of the auxiliary band 200 may be connected to one end of the base band 100 .

[0070] The auxiliary band 200 separates the controller case 300 from the baseband 100, thereby preventing the vibrations of the actuators located in the baseband 100 from being transmitted to the controller case 300, thereby preventing noise. In addition, the auxiliary band 200 has the advantage that the position of the controller case 300 can be adjusted to fit the user's body type by adjusting the relative attachment positions of the detachable member on the back and the detachable member 110.

[0071] The detachable member 210 is located at the front of the auxiliary band 200, and may be detachable from the detachable member 120. The detachable member 210 may be a type A detachable member. Therefore, when a user wears the tactile stimulus providing device 10, the detachable member 120 can be attached to and detached from not only the detachable member 110 but also the detachable member 210, so that the tactile stimulus providing device 10 can be adapted to a wider variety of body types of users.

[0072] According to an embodiment, the width of the other end of the auxiliary band 200 may be greater than the width of the inner circumferential surface of the opening of the controller case 300. For example, the other end of the auxiliary band 200 may be horn-shaped or hook-shaped. Therefore, the controller case 300 may not easily come off the auxiliary band 200.

[0073] The ring 400 may be connected to one end of the baseband 100. In this case, the width of the other end of the baseband 100 may be greater than the width of the inner circumference of the ring. For example, the other end of the baseband 100 may be trumpet-shaped or hook-shaped. According to this embodiment, when the user removes the tactile stimulation providing device 10 from the arm, the other end of the baseband 100 is hooked on the ring 400, so that the device can be prevented from suddenly dropping. In addition, by keeping the other end of the baseband 100 hooked on the ring 400, when the next user wears the tactile stimulation providing device 10, the procedure of reinserting the other end of the baseband 100 into the ring 400 can be omitted. Since the baseband 100 is elastic, the user can bend the other end of the baseband 100 to pass it through the ring 400 when wearing the tactile stimulation providing device 10.

[0074] The top cases 500c may be exposed to the outside, and the top cases 500c exposed to the outside may be in close contact with the user's arm. The top cases 500c transmit the driving force of the corresponding actuator to the user's body, allowing the user to feel a tactile stimulus.

[0075] The controller is located inside the controller case 300 and can generate drive signals to control a plurality of actuators. The controller may be configured to include a microcontroller, a motor driver, a power management module, and the like. There may be a separate battery inside the controller case 300 for driving the controller. The controller may be configured in the form of a PCB, an FPCB, an IC, and the like. In addition, the controller may receive an input of a tactile stimulation pattern through an existing wireless communication technology such as Bluetooth or Wi-Fi or a wired communication technology. The tactile stimulation pattern may be a multi-directional tactile stimulation pattern. Another memory element may be located inside the controller case 300.

[0076] The signal transmission member can transmit a drive signal from the controller to the actuators. The signal transmission member may be made of a material such as FPCB or wire. The signal transmission member can electrically connect the actuators and the controller. The base band 100, the auxiliary band 200, and the controller case 300 may each include an opening through which the signal transmission member can pass.

[0077] FIG. 6 is a diagram showing the tactile stimulus providing device of FIG. 4 being worn by a user.

[0078] 6, the user is wearing the tactile stimulus providing device 10a on his / her body. In FIG. 6, the tactile stimulus providing device 10a is placed on the user's body, and one end of the tactile stimulus providing device 10a including the detachable member 120 has passed through the ring 400.

[0079] FIG. 7 is a diagram showing a state in which the tactile stimulus providing device of FIG. 4 is worn by a user.

[0080] 7, the user fixes the tactile stimulus providing device 10 to the user's body by attaching the detachable member 120 of the tactile stimulus providing device 10a to the detachable member 210. If the part of the body to which the user wears the device is thicker, the detachable member 120 may be attached to the detachable member 110. Therefore, the detachable member 120 can be attached to the detachable member 110 or the detachable member 210, and thus can be adapted to various user bodies.

[0081] FIG. 8 is a view for explaining an actuator case according to another embodiment of the present invention from one perspective, and FIG. 9 is a view for explaining the actuator case of FIG. 8 from another perspective.

[0082] 8 and 9, an actuator case 2 according to another embodiment of the present invention includes a bottom case bc2, a cover case cc2, and a top case tc2.

[0083] The positional relationship between the bottom case bc2, the cover case cc2, and the top case tc2 may be determined based on the baseband. The baseband may be located between the cover case cc2 and the top case tc2. For the relative positional relationship between the baseband and the actuator case 2, further refer to the descriptions of FIGS. 10 to 12.

[0084] The actuator ac2 is located on one side of the baseband and includes a driver mb2 and an eccentric mass em2. The driver mb2 and the eccentric mass em2 may be rotatably connected via a shaft. The actuator ac2 may be an ERM motor. The actuator ac2 can generate vibrations by having the eccentric mass em2 rotated by the driver mb2 act as an unbalanced oscillator. The actuator ac2 may be a cylindrical vibration motor with a cylinder diameter of about 6 mm and a total height of about 17 mm. In a coin-type motor, the oscillator rotates horizontally to generate horizontal vibrations, but the actuator ac2 of this embodiment can generate vertical vibrations.

[0085] The bottom case bc2 is located on one side of the baseband and includes a first housing portion tha2 that houses the driver mb2 and a second housing portion thb2 that houses the eccentric mass em2. That is, the bottom case bc2 can house all or a part of the actuator ac2 except for the wiring. Depending on the embodiment, when the bottom case bc2 houses only a part of the actuator ac2, the remaining part of the actuator ac2 may be housed by a cover case cc2 described later.

[0086] The cover case cc2 is located on one side of the baseband and is coupled to the bottom case bc2 so as to bring the driving unit mb2 into close contact with the first receiving portion tha2.

[0087] 8 and 9, the cover case cc2 may be coupled to the bottom case bc2 while covering the actuator ac2. The male fastening portion cm2 of the cover case cc2 may be fitted into the female fastening portion bsf2 of the bottom case bc2, so that the cover case cc2 may be firmly fixed to the bottom case bc2. In particular, a first receiving portion tha2 and a receiving portion cha2 may be formed so that the driving portion mb2 of the actuator ac2 may be firmly fixed. The male fastening portion cm2 of the cover case cc2 may protrude parallel to a surface where the cover case cc2 abuts against the baseband. In this embodiment, there are four pairs of male fastening portions cm2 and female fastening portions bsf2, but the number may vary depending on the embodiment.

[0088] The cover case cc2 is coupled to the bottom case bc2, isolating the eccentric mass em2 from one side of the baseband, and together with the second housing portion thb2, is able to form a generous rotation space for the eccentric mass em2. That is, the housing portion chb2 of the cover case cc2 is matched with the second housing portion thb2, and is able to form a generous rotation space that does not impede the rotation of the eccentric mass em2. The housing portion chb2 of the cover case cc2 also serves to isolate the eccentric mass em2 from one side of the baseband. For this reason, even if the baseband is made of a flexible material such as cloth or rubber, the eccentric mass em2 can rotate reliably without the risk of hitting the baseband.

[0089] The top case tc2 is located on the other side of the baseband and fixes the bottom case bc2 to the baseband.

[0090] The male fastening portion tm2 of the top case tc2 penetrates the base band and is coupled to the female fastening portion bf2 of the bottom case, thereby fixing the actuator ac2 and the actuator case 2 to the base band. At this time, the base band may include an opening shaped to allow the male fastening portion tm2 to pass through. Depending on the embodiment, the area of ​​the top case tc2 that is in close contact with the other side of the base band may correspond to the area of ​​the bottom case bc2 that is in close contact with one side of the base band. In this case, since the space in which the male fastening portion tm2 can be located is relatively sufficient, in this embodiment, the male fastening portion tm2 and the female fastening portion bf2 are formed in four pairs. Therefore, even if the actuator ac2 vibrates more strongly, it can withstand this. According to this embodiment, when the area of ​​the top case tc2 corresponds to the area of ​​the bottom case bc2, sufficient vibration force is transmitted to the back of a user with a relatively low sensitivity, so that the user can feel the vibration more reliably. In addition, in this embodiment, the corners of one side of the top case tc2 are rounded to provide a more comfortable feeling to the user.

[0091] The bottom case bc2 may include a protrusion bm2, and the top case tc2 may include a recess tf2. The protrusion bm2 of the bottom case bc2 presses against the base band and fits into the recess tf2, so that the actuator ac2 and the actuator case 2 can be more firmly fixed to the base band. In this embodiment, since the area of ​​the top case tc2 is relatively sufficient, six pairs of the protrusion bm2 and the recess tf2 are formed accordingly to provide sufficient support.

[0092] Figure 10 is a front view of a vest-type tactile stimulus providing device according to one embodiment of the present invention, Figure 11 is a rear view of the tactile stimulus providing device of Figure 10, and Figure 12 is a view for explaining the case where the front panel of the tactile stimulus providing device of Figure 11 is opened.

[0093] 10 to 12, the tactile stimulation providing device 10b may include a first front panel rp1, a second front panel lp1, and a rear panel bp1. Each of the panels rp1, lp1, and bp1 may be made up of a plurality of layers, and at least one of the plurality of layers may correspond to a baseband. One side of the baseband may be located inside the panel and not be visible. Referring to FIG. 12, the other side of the baseband may be exposed to the inside.

[0094] The first front panel rp1 may include a plurality of actuators arranged between one side and the other side. The second front panel lp1 may include a plurality of actuators arranged between one side and the other side. One side of the second front panel lp1 may be fastenable to the other side of the first front panel rp1. For example, one side of the second front panel lp1 and the other side of the first front panel rp1 may be fastenable to each other by a zipper, Velcro, a buckle, a button, or the like.

[0095] The rear panel bp1 may include a plurality of actuators arranged between one side and the other side, one side of which may be fastened to one side of the first front panel rp1 and the other side of which may be fastened to the other side of the second front panel lp1.

[0096] The rear panel bp1 may include a first ring rr1 connected to one side and a second ring lr1 connected to the other side. The first front panel rp1 may include a first wing band rwb1 connected to one side. The second front panel lp1 may include a second wing band lwb1 connected to the other side.

[0097] The detachable member rv21 may be located at one end of the front surface of the first wing band rwb1. The detachable member rv11 may be located at the front surface of the first front panel rp1 and may be detachable from the detachable member rv21.

[0098] The detachable member lv21 may be located at one end of the front surface of the second wing band lwb1. The detachable member lv11 may be located at the front surface of the second front panel lp1 and may be detachable from the detachable member lv21.

[0099] The user can fit the first front panel rp1 to the user's torso by passing one end of the first wing band rwb1 through the first ring rr1, and then pulling it according to the size of the body to attach the detachable member rv21 to the detachable member rv11. Similarly, the user can fit the second front panel lp1 to the user's torso by passing one end of the second wing band lwb1 through the second ring lr1, and then pulling it according to the size of the body to attach the detachable member lv21 to the detachable member lv11.

[0100] According to an embodiment, the width of one end of the first wing band rwb1 may be greater than the width of the inner circumferential surface of the first ring rr1, and the width of one end of the second wing band lwb1 may be greater than the width of the inner circumferential surface of the second ring lr1. For example, one end of the first wing band rwb1 and one end of the second wing band lwb1 may be configured in a trumpet or hook shape. Therefore, when a user removes the tactile stimulation providing device 21 from the body, one end of the wing bands rwb1, lwb1 is kept hooked on each of the rings rr1, lr1, so that when the next user wears the tactile stimulation providing device 21, the procedure of reinserting the wing bands rwb1, lwb1 into each of the rings rr1, lr1 can be omitted.

[0101] The back panel bp1 may further include a pocket bpp for battery storage or other uses. A portion of the pocket bpp may have an opening to allow the battery wires to extend inside the panels lp1, rp1, bp1. In other embodiments, the controller may use an internal battery. In this case, the pocket bpp may not be necessary.

[0102] Referring to FIG. 12, the right front panel rp1, the back panel bp1, and the left front panel lp1 of a tactile stimulation providing device 10b according to another embodiment of the present invention may each include a plurality of actuators.

[0103] When the front panels rp1, lp1 are opened as in FIG. 12, a plurality of top cases tcr1 of the first front panel rp1, a plurality of top cases tcb1 of the rear panel bp1, and a plurality of top cases tcl1 of the second front panel lp1 may be exposed.

[0104] The tactile stimulation providing device 10b is fastened to the user's torso by the wing bands lwb1, rwb1, the detachable members rv11, rv21, lv11, lv21, and the rings lr1, rr1, so that the top cases tcr1, tcb1, tcl1 can be tightly attached to the user's torso. Therefore, the user can strongly feel the vibration of the built-in actuators through the tightly attached top cases tcr1, tcb1, tcl1.

[0105] In another embodiment, the inner skin acting as the lining for the panels rp1, bp1, lp1 may cover the top cases tcr1, tcb1, tcl1 so that the top cases tcr1, tcb1, tcl1 are not exposed to the outside.

[0106] Depending on the embodiment, the first front panel rp1 may be elastically connected to the rear panel bp1 by a connecting band rcb1, and the second front panel lp1 may be elastically connected to the rear panel bp1 by a connecting band lcb1.

[0107] The first front panel rp1 may include actuators corresponding to the top case tcr1. For example, a column FC1 of front actuators may include actuators ac11, ac12, ac13, ac14, and ac15 arranged in a column on one side of the first front panel rp1. Also, a column FC2 of front actuators may include actuators arranged in a column on the other side of the first front panel rp1.

[0108] The second front panel lp1 may include actuators corresponding to the top case tcl1. For example, a front actuator column FC3 may include actuators arranged in a column on one side of the second front panel lp1. Also, a front actuator column FC4 may include actuators ac41, ac42, ac43, ac44, and ac45 arranged in a column on the other side of the second front panel lp1.

[0109] The rear panel bp1 may include actuators corresponding to the top case tcb1. For example, a row of rear actuators BC1 may include actuators arranged in a row on one side of the rear panel bp1. A row of rear actuators BC4 may include actuators arranged in a row on the other side of the rear panel bp1. The rows of rear actuators BC2 and BC3 may be located between the rows of rear actuators BC1 and BC4.

[0110] When the user wears the haptic sense providing device 10b, the front actuator column FC1 and the rear actuator column BC1 are located on the right side of the user, and the front actuator column FC4 and the rear actuator column BC4 are located on the left side of the user. Also, when the user wears the haptic sense providing device 10b, the front actuator columns FC2 and FC3 are located in the center of the front of the user, and the rear actuator columns BC2 and BC4 are located in the center of the rear of the user.

[0111] FIG. 13 is a diagram for explaining a user interface according to an embodiment of the present invention.

[0112] The user interface 70a may be generated by a program, which may be executed on a computer, and which may be recorded on a computer-readable recording medium.

[0113] 13, user interface 70a may include a mode selection area 71, a channel selection area 72, a global settings area 73, a simulation area 74, a channel settings area 75, and a local settings area 76. Depending on the embodiment, some areas may be excluded from user interface 70a.

[0114] The mode selection area 71 allows the selection of one of a stereo audio mode having two audio channels and a surround audio mode having three or more audio channels. In Fig. 13, only a button corresponding to the stereo audio mode and a button corresponding to the 7.1 channel surround audio mode are shown, but in other embodiments, a button corresponding to the 5.1 channel surround audio mode may be added. In the following, it is assumed that the stereo audio mode is selected in the mode selection area 71.

[0115] The channel selection area 72 may be an area in which one of the channels of the selected audio mode can be selected. For example, a stereo audio mode may include a first channel ch1a and a second channel ch2a. In the following, for convenience of explanation, it is assumed that the first channel ch1a is a channel of a speaker located to the right of the user, and the second channel ch2a is a channel of a speaker located to the left of the user. It is also assumed that the first channel ch1a is selected.

[0116] The global setting area 73 may include global setting values ​​that are commonly applied to all frequency bands. For example, the global setting values ​​may include a global maximum threshold GHTH, a global minimum threshold GLTH, and a global gain value GGA. A user can easily adjust the maximum threshold, minimum threshold, and gain values ​​for all frequency bands (or a selected frequency band) by adjusting the global maximum threshold GHTH, the global minimum threshold GLTH, and the global gain value GGA.

[0117] The simulation area 74 can visualize the drive strength of the virtual actuators in response to the input audio data. For example, the size of the graphic symbol displayed on the virtual actuator may change depending on the magnitude of the drive strength. For example, if the drive strength is large, the graphic symbol may be displayed large, and if the drive strength is small, the graphic symbol may be displayed small. Therefore, the user can intuitively recognize the magnitude of the vibration force of the corresponding actual actuator based on the size of the graphic symbol for each virtual actuator.

[0118] The channel setting area 75 may include a frequency band of at least one of the two or more audio channels. For example, when the first channel ch1a is selected in the channel selection area 72, the frequency bands SFT11 to SFT115 of the first channel ch1a may be displayed. For example, the ranges of the frequency bands SFT11 to SFT115 may be default values ​​or values ​​set by the user. For example, the user may activate or deactivate each frequency band using the button SFB. Although not shown, the channel setting area 75 may further include a button for determining whether or not to apply a global setting value to the corresponding frequency band.

[0119] 13, the channel setting area 75 displays only the frequency band SFT11-SFT115 of the first channel ch1a, but in other embodiments, the channel setting area 75 may display all of the frequency bands of the first channel ch1a and the second channel ch2a. In such embodiments, the channel selection area 72 may be omitted from the user interface 70a.

[0120] The local setting area 76 may include a local setting value for a selected one of the frequency bands. The local setting value may include a virtual actuator selected from the plurality of virtual actuators corresponding to the selected frequency band. The local setting value may include a maximum threshold value that is an upper limit of valid data for the selected frequency band, and a minimum threshold value that is a lower limit of valid data. The local setting value may also include a gain value that is a scaling factor for data for the selected frequency band.

[0121] For example, when the user selects the frequency band SFT11 from the frequency bands SFT11 to SFT115, the user can set a maximum threshold SHTH11 and a minimum threshold SLTH11 for the frequency band SFT11 in the local setting area 76. The user can also set a gain value SFG11 for the frequency band SFT11 in the local setting area 76.

[0122] When the user sets a global setting value in the global setting area 73, the global setting value may be applied as a reference value, and the local setting value may be applied as an offset value. That is, the sum of the global maximum threshold GHTH and the maximum threshold SHTH11 may be the final maximum threshold for the frequency band SFT11. Similarly, the sum of the global minimum threshold GLTH and the minimum threshold SHTH11 may be the final minimum threshold for the frequency band SFT11. Similarly, the sum of the global gain value GGA and the gain value SFG11 may be the final gain value for the frequency band SFT11.

[0123] In other embodiments, the global settings may be ignored when a user sets the maximum threshold SHTH11, minimum threshold SLTH11, and gain value SFG11 for frequency band SFT11 in local settings area 76. In such embodiments, the final settings for frequency band SFT11 may be the maximum threshold SHTH11, minimum threshold SLTH11, and gain value SFG11, regardless of the global settings.

[0124] Virtual actuators va11 to va45 corresponding to actuators ac11 to ac45 in Fig. 12 may be displayed in the local setting area 76. Columns VFC1, VFC2, VFC3, and VFC4 of the virtual actuators may correspond to columns FC1, FC2, FC3, and FC4 of the front actuators in Fig. 12. Similarly, columns VBC1, VBC2, VBC3, and VBC4 of the virtual actuators may correspond to columns BC1, BC2, BC3, and BC4 of the rear actuators in Fig. 12.

[0125] The user can select virtual actuators va11 to va45 corresponding to the frequency band SFT11 in the local setting area 76. The actuators ac11, ac12, and ac13 corresponding to the selected virtual actuators va11, va12, and va13 may be driven with drive strengths corresponding to the received actuator strength values.

[0126] The user interface 70a may also include various other settings for the convenience of the user. For example, depending on the state of the buttons AAB, it may be determined whether the local settings of the selected frequency band are applied to other frequency bands as well. This eliminates the need for the user to perform local settings for all frequency bands one by one, improving user convenience.

[0127] For example, the automatic setting unit 96 in Fig. 25 may determine whether or not to operate depending on the state of the button ADB. For the automatic setting unit 96, see the description of Fig. 25.

[0128] For example, when the button UCB is in a first state (locked state), the virtual actuators in the local setting area 76 can be in an unselectable state. In this case, a virtual actuator determined as a default for the selected frequency band may be fixedly selected. When the button UCB is in a second state (unlocked state), the virtual actuators in the local setting area 76 can be in a selectable state. In this case, the user can select a virtual actuator for the selected frequency band, as described above.

[0129] For example, the state of the button FHB may be dependent on the state of the button UCB. For example, when the button UCB is in a first state (locked state), the button FHB may be activated. At this time, when the activated button FHB is in the first state, the lower the frequency band, the higher the actuator row may be mapped, and the higher the frequency band, the lower the actuator row may be mapped. On the other hand, when the activated button FHB is in a second state, the higher the frequency band, the higher the actuator row may be mapped, and the lower the frequency band, the lower the actuator row may be mapped. This eliminates the need for the user to manually map the actuators to the frequency bands one by one, improving user convenience. When the button UCB is in a second state (unlocked state), the button FHB may be inactivated.

[0130] Although not shown, a button for copying the setting value of the first channel ch1a as the setting value of the second channel ch2a may be added. In this case, the row of virtual actuators corresponding to the setting value of the first channel ch1a may be different from the row of virtual actuators corresponding to the copied setting value of the second channel ch2a. For example, if the first channel ch1a corresponds to a speaker located to the right of the user and the second channel ch2a corresponds to a speaker located to the left of the user, the row of virtual actuators corresponding to the setting value of the first channel ch1a may be symmetrical (left-right reversed) with the row of virtual actuators corresponding to the copied setting value of the second channel ch2a. In other embodiments, this button does not exist, and the function may be included as a basic function of the user interface 70a.

[0131] According to the above embodiment, even if a setting value for only one channel is set, setting values ​​for other channels are also set, so that convenience for the user can be improved.

[0132] 14 and 15 are diagrams for explaining the simulation area of ​​the user interface of FIG.

[0133] Referring to FIG. 14, a simulation domain 74' according to another embodiment is shown.

[0134] The simulation area 74' may include an area 742 displaying the frequency band of the selected channel, and an area 741 displaying the signal strength for each frequency. For example, the signal strength for each frequency may be displayed as a power spectral density (PSD), magnitude, phase, etc. Also, the area 741 may display a graph of a representative value RPV for each frequency band.

[0135] According to the present embodiment, the simulation domain 74' can convert each frame of currently played audio data into the frequency domain in real time and display it in decibels. By referring to this information, the user can easily set the gain value, minimum threshold value, and maximum threshold value of each frequency band.

[0136] Referring to FIG. 15, a simulation domain 74'' according to another embodiment is shown.

[0137] The simulation area 74'' can visualize the frequency distribution of the representative values ​​over time for each of the frequency bands SFT11 to SFT110. For example, in each frequency band, a brighter area can mean a larger number of frames having the corresponding representative value, and a darker area can mean a smaller number of frames having the corresponding representative value.

[0138] Therefore, the user can easily set the gain value, minimum threshold value, and maximum threshold value for each frequency band by referring to the simulation domain 74''.

[0139] Figure 16 is a diagram for explaining a tactile stimulation pattern generating device according to one embodiment of the present invention, Figure 17 is a diagram for explaining the data conversion section of the tactile stimulation pattern generating device of Figure 16, Figure 18 is a diagram for explaining the band representative value calculation section of the tactile stimulation pattern generating device of Figure 16, Figures 19 to 22 are diagrams for explaining the actuator strength calculation section of the tactile stimulation pattern generating device of Figure 16, and Figure 23 is a diagram for explaining the actuator strength merging section of the tactile stimulation pattern generating device of Figure 16.

[0140] Referring to Figure 16, a tactile stimulation pattern generating device 9 according to one embodiment of the present invention may include a data conversion unit 91, a band representative value calculation unit 92, an actuator strength calculation unit 93, an actuator strength attenuation unit 94, and an actuator strength merging unit 95.

[0141] The data conversion unit 91 can convert the first audio data IN1a and the second audio data IN2a in the time domain into first converted data and second converted data in the frequency domain, respectively.

[0142] In one embodiment, the first audio data IN1a and the second audio data IN2a may be audio data directly received by the tactile stimulation pattern generating device 8. Alternatively, the first audio data IN1a and the second audio data IN2a may be audio data stored in a recording medium.

[0143] In another embodiment, the first audio data IN1a and the second audio data IN2a may be audio data transmitted from a first program (e.g., a game program) to an audio device (e.g., a speaker, a headset, etc.) and intercepted by the tactile stimulation pattern generating device 9. The first program may be a computer program, and the tactile stimulation pattern generating device 9 may be embodied as a second program of the computer. For example, the tactile stimulation pattern generating device 9 may generate the first audio data IN1a and the second audio data IN2a by intercepting a system call generated from an operating system of the computer.

[0144] In this embodiment, it is assumed that the audio data is a stereo audio mode having two channels, but in other embodiments for a surround audio mode having three or more channels, the audio data may be three or more. In addition, only the first converted data FFI corresponding to the first audio data IN1a will be described below. The same content can be applied to the second converted data corresponding to the second audio data IN2a, and the overlapping description will be omitted.

[0145] 16 and 17, the data conversion unit 91 can sample a plurality of frames FR1, FR2, FR3, and FR4 from the first audio data IN1a based on the window length WDL and the shift length SHFL.

[0146] The window length WDL is a setting value that determines the length of the frames FR1, FR2, FR3, and FR4 on the time axis in the first audio data IN1a. The longer the window length WDL, the lower the frequency components that can be detected and included in each of the frames FR1 to FR4, and the higher the frequency resolution. However, the longer the window length WDL, the greater the data amount and sampling time, so the window length WDL must be set appropriately. Also, if the window length WDL is set too long, the change in the audio data may not be properly reflected in the representative value RPV. For example, when the window length WDL is 500 ms and the signal strength increases due to a sound effect such as a gunshot in the last part of the window (e.g., 450 ms to 500 ms), even if the signal strength increases significantly, the change occurs only in 10% of the entire window, and therefore may not be properly reflected in the representative value RPV.

[0147] For example, the window length WDL can be set relatively long in a portion where the time signature, tempo, beat, etc. of a song or musical performance are important. Also, the window length WDL can be set relatively short in a portion where sound effects such as gunshots and popping sounds are important. Such a window length WDL can be set automatically by analyzing the audio data or can be set directly by the user. Depending on the embodiment, the window length WDL can be varied according to time even within one piece of audio data.

[0148] The shift length SHFL is a setting value that determines the position of the next frame on the time axis based on the position of the current frame on the time axis. That is, the shift length SHFL is a setting value for the time interval between the current frame and the next frame. The degree of overlapping of the current frame and the next frame may be determined by the window length WDL and the shift length SHFL. If the shift length SHFL is too long, the user may feel a latency, so an appropriate setting value is required.

[0149] Although not shown, the data conversion unit 91 may include a sampling frequency as a set value. In the case of digital audio data, the system and the file have a preset sampling frequency, so the data conversion unit 91 can use that sampling frequency. However, when receiving analog audio data such as an audio jack, the data conversion unit 91 needs to receive an appropriate sampling frequency. In this case, the sampling frequency may be determined according to the amount of data that can be processed and the associated speed, as well as the characteristics of the audio signal (higher sampling frequency for important sounds such as detailed melodies, and lower frequency and faster for short sounds that spread over a wide band such as gunfires), similar to the window length WDL.

[0150] According to an embodiment, the data converter 91 can apply a correction window to the sampled frame FR1. The correction window may be a conventionally known window used in a Fast Fourier Transform (FFT), such as a Hann, Hamming, or Blackmann-Harris window. By applying the correction window to the sampled frame FR1, the data converter 91 can prevent FFT spectrum leakage caused by discontinuity of the start data and end data of the frame, and can obtain more accurate intensity information for frequencies.

[0151] The data converter 91 can generate the first transformed data FFI by transforming the frame FR1 to which the correction window has been applied into the frequency domain. For the transformation into the frequency domain, various existing algorithms such as the above-mentioned FFT may be used.

[0152] The band representative value calculation unit 92 can calculate a representative value RPV of the frequency bands SFT11 to SFT110 for each of the first converted data FFI and the second converted data. The band representative value calculation unit 92 can further receive a minimum threshold SLTH, a maximum threshold SHTH, and a gain value SFG. In this case, the representative value RPV may be based on valid data that is greater than the minimum threshold SLTH and less than the maximum threshold SHTH for each of the frequency bands SFT11 to SFT110.

[0153] 18, at least two of the frequency bands SFT11, SFT12, SFT13, and SFT14 may overlap at least partially. For example, the frequency bands SFT11 and SFT12 may overlap, and the frequency bands SFT11 and SFT13 may overlap. Furthermore, the frequency band SFT14 may partially overlap the frequency bands SFT11 and SFT13. This makes it possible to effectively cover sounds in a relatively wide frequency range, such as drum beats and bass sounds, while emphasizing sounds in a relatively narrow frequency range, such as voices (acoustic characteristic emphasis effect).

[0154] According to this configuration, the representative values ​​of the frequency band SFT11 covering a wide frequency range are converted into a relatively large number of actuator strength values, and the representative values ​​of the frequency bands SFT12 and SFT13 covering narrow frequency ranges can be converted into a relatively small number or one actuator strength value, which makes the user feel as if he or she is in front of a larger speaker, resulting in a stronger sense of immersion.

[0155] According to an embodiment, a first minimum threshold SLTH11 corresponding to a first frequency band SFT11 among the frequency bands SFT11-SFT14 may be different from a second minimum threshold SLTH12 corresponding to a second frequency band SFT12 among the frequency bands SFT11-SFT14. Also, a first maximum threshold SHTH11 corresponding to the first frequency band SFT11 may be different from a second maximum threshold SHTH12 corresponding to the second frequency band SFT12. According to an embodiment, a first gain value SFG11 corresponding to the first frequency band SFT11 among the frequency bands SFT11-SFT14 may be different from a second gain value corresponding to the second frequency band SFT12 among the frequency bands SFT11-SFT14.

[0156] In another embodiment, the minimum threshold SLTH, the maximum threshold SHTH, and the gain value SFG may be set commonly for the frequency bands SFT11 to SFT14 (see the global setting region 73 in FIG. 13).

[0157] First, the band representative value calculation unit 92 can apply a first gain value SFG11 to the data (intensity value) of the first frequency band SFT11. For example, when the data and the first gain value SFG11 are values ​​on a linear scale, each of the data can be multiplied by the first gain value SFG11. For example, when the data and the first gain value SFG11 are values ​​on a logarithmic scale, each of the data can be summed by the first gain value SFG11. In FIG. 18, for convenience of explanation, it is assumed that the gain value of each of the frequency bands SFT11 to SFT14 is 0 (logarithmic scale). Similarly, the band representative value calculation unit 92 can apply a second gain value to the data (intensity value) of the second frequency band SFT12.

[0158] Next, the band representative value calculation unit 92 can determine, among the data (intensity values) of the first frequency band SFT11, data that is greater than the first minimum threshold SLTH11 and smaller than the first maximum threshold SHTH11 as valid data of the first frequency band SFT11. Similarly, the band representative value calculation unit 92 can determine, among the data (intensity values) of the second frequency band SFT12, data that is greater than the second minimum threshold SLTH12 and smaller than the second maximum threshold SHTH12 as valid data of the second frequency band SFT12.

[0159] Finally, the band representative value calculation unit 92 can determine a representative value RPV of the first frequency band SFT11 based on the valid data of the first frequency band SFT11. Similarly, the band representative value calculation unit 92 can determine a representative value of the second frequency band SFT12 based on the valid data of the second frequency band SFT12.

[0160] The representative value RPV may be set as an average value of the valid data in each frequency band, an RMS (root mean square) of the valid data, a total value of the valid data, etc. The valid data may be at least one of a PSD value, a magnitude value, and a phase value.

[0161] The actuator strength calculation unit 93 can calculate the actuator strength value MDP corresponding to the representative value RPV. With reference to Fig. 19 to Fig. 22, various examples of a mapping curve CCV of the actuator strength value MDP to the representative value RPV are shown.

[0162] According to one embodiment, the actuator strength value MDP can be increased as the representative value RPV increases (the sound becomes louder) to provide the user with an appropriate haptic stimulus. However, when the representative value RPV is close to the upper limit and the size of the effective data increases further, the representative value RPV output by the band representative value calculation unit 92 may become 0. In this case, there is a problem that the provision of the haptic stimulus may be suddenly interrupted during the provision of a strong haptic stimulus. In such a case, the user may feel inconvenienced by the strong haptic stimulus being provided intermittently.

[0163] 19, the actuator strength calculation unit 93 can calculate the actuator strength value MDP to be smaller as the representative value RPV is smaller, and calculate the actuator strength value MDP to be smaller as the representative value RPV is larger. In one embodiment, at least one representative value RPV can correspond to a maximum value MMX of the actuator strength values ​​MDP.

[0164] 20, the relationship between the representative value RPV and the actuator strength value MDP may include a discontinuous function. For example, the mapping curve CCV may be non-linear. For example, the mapping curve CCV may be set by a classification algorithm using machine learning or the like.

[0165] 21, it can be seen that the mapping curve CCV can be set to a curved shape. For example, the mapping curve CCV may include logarithmic functions that are symmetrical with respect to the representative value RPV corresponding to the maximum value MMX.

[0166] 22, the mapping curve CCV may be set so that the representative value RPV corresponding to the maximum value MMX is closer to the maximum threshold SHTH than the minimum threshold SLTH. According to this embodiment, the actuator strength value MDP can be naturally increased as the representative value RPV increases (as the sound becomes louder). Furthermore, if the representative value RPV can be continuously provided near the maximum threshold SHTH, intermittent provision of strong tactile stimulation can be prevented.

[0167] The actuator strength attenuation unit 94 may provide an attenuated actuator strength value AMDP by gradually attenuating the actuator strength value MDP when the actuator strength value MDP is held for a certain period of time or more.

[0168] A user may feel uncomfortable if the same vibration continues. In addition, human tactile sense has a characteristic that it reacts less to the same stimulus provided continuously. Therefore, if the actuator strength value MDP is held for a certain period of time or more, an option to gradually attenuate the actuator strength value MDP may be provided to the user.

[0169] The actuator strength attenuation unit 94 can exponentially reduce the actuator strength value MDP (see Equation 1).

[0170]

number

[0171] In this case, y(t) is the actuator strength value MDP at the current time, y(t-1) is the actuator strength value MDP at the immediately previous time, y'(t) is the attenuated actuator strength value AMDP at the current time, y'(t-1) is the attenuated actuator strength value AMDP at the previous time, a is a weighting value for the change value, f is the frequency median of the corresponding frequency band, tau is a attenuation parameter, and dt is a sampling interval (e.g., shift length SHFL).

[0172] After a time equivalent to tau cycles of a sine wave having a frequency of f, the actuator strength value MDP can be halved. According to Equation 1, since y'(t) may be smaller than y(t) in most circumstances depending on the set value of a, y'(t) needs to be periodically reset to y(t). For example, when the absolute value of the change in the actuator strength value MDP (y(t)-y(t-1)) is greater than a certain value, y'(t) needs to be initialized to y(t).

[0173] Although the actuator strength attenuation unit 94 in the above-described embodiment attenuates the actuator strength value MDP, in other embodiments, the actuator strength attenuation unit 94 can also attenuate the representative value RPV. For example, the actuator strength attenuation unit 94 can gradually attenuate the representative value RPV when the representative value RPV is held for a certain period of time or more.

[0174] The actuator strength merging unit 95 can generate one actuator strength value by merging two or more actuator strength values ​​mapped to one actuator.

[0175] Referring to FIG. 23, for example, the actuator strength value AMDP11 of frequency band SFT11 may be mapped to actuators ac11, ac12, and ac13, the actuator strength value AMDP12 of frequency band SFT12 may be mapped to actuator ac11, and the actuator strength value AMDP13 of frequency band SFT13 may be mapped to actuator ac12.

[0176] The merging of actuator strength values ​​AMDP11 and AMDP12 is necessary because actuator ac11 is driven only for one actuator strength value (see Figures 12 and 13).

[0177] According to one embodiment, an average value of the actuator strength values ​​AMDP11 and AMDP12 may be provided to the actuator ac11 as a combined actuator strength value. According to another embodiment, the actuator strength values ​​AMDP11 and AMDP12 may be multiplied by different weighting values, and the sum of the multiplied values ​​may be provided to the actuator ac11 as a combined actuator strength value. According to yet another embodiment, the largest value of the actuator strength values ​​AMDP11 and AMDP12 may be provided to the actuator ac11 as a combined actuator strength value.

[0178] If the actuator strength value AMDP is a logarithmic scale value such as decibels, the actuator strength value AMDP can first be converted to a linear scale using an exponential function. After merging the linear scale actuator strength values ​​AMDP, the merged actuator strength value can be converted back to a logarithmic scale and provided to the actuator. This process allows the user to feel the merged actuator strength value in a natural way.

[0179] FIG. 24 is a diagram for explaining an exemplary relationship between a tactile stimulation pattern generating device and a tactile stimulation providing device.

[0180] 24, when an audio mode includes eight channels ch1b to ch8b, such as a 7.1 surround audio mode, there is shown an example of how the channels are mapped to actuator columns FC1 to FC4 and BC1 to BC4.

[0181] For example, the tactile stimulus providing device 10b may provide position information corresponding to each actuator. The position information may be information regarding at which part of the tactile stimulus providing device 10b each actuator is located. For example, the tactile stimulus pattern generating device 9 may provide position information corresponding to each virtual actuator. Depending on the embodiment, at least one of the tactile stimulus providing device 10b and the tactile stimulus pattern generating device 9 may provide the above-mentioned position information.

[0182] For example, the audio data for channels ch1b to ch8b may include directional information for the corresponding channels.

[0183] According to one embodiment, the first audio data for channel ch6b may include information regarding a first direction (right direction), and the second audio data for channel ch5b may include information regarding a second direction (left direction) different from the first direction.

[0184] At this time, an actuator (one of the actuators in the front actuator column FC1 and the rear actuator column BC1) corresponding to the first position information corresponding to the first direction may be driven based on at least one of the first actuator strength values ​​of the first audio data (not based on the second audio data), and an actuator (one of the actuators in the front actuator column FC4 and the rear actuator column BC4) corresponding to the second position information corresponding to the second direction may be driven based on at least one of the second actuator strength values ​​of the second audio data (not based on the first audio data).

[0185] Similarly, an actuator column corresponding to a first position information corresponding to a first direction (one of the column of front actuators FC1 and the column of rear actuators BC1) may be driven based on only at least one of the first actuator strength values ​​of the first audio data, and an actuator column corresponding to a second position information corresponding to a second direction (one of the column of front actuators FC4 and the column of rear actuators BC4) may be driven based on only at least one of the second actuator strength values ​​of the second audio data.

[0186] For example, channel ch1b corresponding to the subwoofer may be mapped to the front actuator columns FC1, FC2, FC3, and FC4. Channel ch2b corresponding to the left side of the front face may be mapped to the front actuator column FC3. Channel ch3b corresponding to the center of the front face may be mapped to the front actuator columns FC2 and FC3. Channel ch4b corresponding to the right side of the front face may be mapped to the front actuator column FC1.

[0187] Additionally, channel ch5b corresponding to the left direction may be mapped to column FC4 of the front actuators and column BC4 of the rear actuators. Channel ch6b corresponding to the right direction may be mapped to column FC1 of the front actuators and column BC1 of the rear actuators. Channel ch7b corresponding to the rear left direction may be mapped to column BC3 of the rear actuators. Channel ch8b corresponding to the rear right direction may be mapped to column BC2 of the rear actuators.

[0188] The above directions are based on the user wearing the tactile stimulus providing device 10. According to the above embodiment, the directionality of the sound can be emphasized depending on the position of the actuator. In particular, even if a 7.1 surround audio system is installed in a specific space, when the user moves or changes direction, the 7.1 surround audio system at a fixed position cannot provide proper directionality of the sound. According to the present embodiment, the directionality of the sound is emphasized through the tactile stimulus providing device 10 worn directly by the user, so that the directionality of the sound will not be inaccurate even if the user moves or changes direction.

[0189] FIG. 25 is a diagram for explaining a tactile stimulation pattern generating device according to another embodiment of the present invention.

[0190] The tactile stimulation pattern generation device 9′ in Fig. 25 differs from the tactile stimulation pattern generation device 9 in Fig. 16 in that it further includes an automatic setting unit 96. The band representative value calculation unit 92 of the tactile stimulation pattern generation device 9′ differs from the band representative value calculation unit 92 of the tactile stimulation pattern generation device 9 in that it uses the maximum threshold value SHTH, the minimum threshold value SLTH, and the gain value SFG provided by the automatic setting unit 96.

[0191] For example, the automatic setting unit 96 can calculate the effective range of the representative value RPV for each frequency band based on the frequency distribution of the representative value RPV over time, set the minimum threshold SLTH to correspond to the minimum value of the effective range, and set the maximum threshold SHTH to correspond to the maximum value of the effective range (see the histogram in Figure 15).

[0192] For example, if the interval between the minimum threshold value SLTH and the maximum threshold value SHTH is too wide, the actuator may provide unnecessary continuous vibration, which may reduce the accuracy of the tactile stimulation. Also, if the interval between the minimum threshold value SLTH and the maximum threshold value SHTH is too narrow, the target sound effect or the like may not be detected, and a meaningful tactile stimulation may not be provided. Therefore, according to this embodiment, by providing an appropriate minimum threshold value SLTH and maximum threshold value SHTH, it is possible to provide not only a meaningful tactile stimulation but also convenience for the user. Also, according to this embodiment, the minimum threshold value SLTH and the maximum threshold value SHTH can be adaptively changed in real time.

[0193] According to one embodiment, the automatic setting unit 96 can change the minimum threshold value SLTH and the maximum threshold value SHTH within a range limited by the minimum threshold value SLTH and the maximum threshold value SHTH preset by the user, thereby preventing sounds intended by the user such as too quiet (e.g., small background sounds) and too loud (e.g., a nearby explosion) from being converted into tactile stimuli.

[0194] Alternatively, the automatic setting unit 96 can provide at least one of the maximum threshold value SHTH, the minimum threshold value SLTH, and the gain value SFG through various algorithms.

[0195] For example, an algorithm may be implemented that compares a statistical value such as the average or RMS of the representative value RPV of a frequency band for the previous 5 seconds with a reference value and adjusts the maximum threshold SHTH and the minimum threshold SLTH according to the difference or ratio. For example, if the average PSD value of a frequency band for the previous 5 seconds is -50 dB and the reference value is -40 dB, the maximum threshold SHTH can be lowered by 10 dB accordingly.

[0196] Also, an algorithm may be implemented to calculate a confidence interval (CI) of this average value, and set the maximum threshold SHTH and the minimum threshold SLTH based on this. For example, when the 95% confidence interval of this average value is calculated to be -45dB to -55dB, if the reference average value is -40dB and the reference 95% confidence interval range is 20dB, the calculated confidence interval range is 10dB, which is twice narrower than the reference range, so the difference between the maximum threshold SHTH and the minimum threshold SLTH is narrowed by two times, and since it is 10dB lower than the reference average, the maximum threshold SHTH and the minimum threshold SLTH can be lowered by 10dB.

[0197] Also, an algorithm may be implemented that uses a Prediction Interval (PI) that allows the distribution of individual sample values ​​to be estimated, rather than a confidence interval that indicates the distribution of the average value. In particular, when the statistical distribution of the input signal cannot be predicted in advance, non-parametric PI estimation may be used. When the non-parametric PI method is used, it is possible to calculate an interval in which the current input can be located with a 95% probability by using the past representative value RPV of each frequency band. For example, when there are n past representative values, in calculating the PI of p (e.g., 0.95 meaning 95%), the n samples are sorted in ascending order, and j and k can be obtained as in the following Equations 2 and 3.

[0198]

number

[0199]

number

[0200] When j and k are calculated in this way, the jth and kth values ​​become the lower and upper limits of the p% prediction interval, respectively. These can be used as the minimum threshold SLTH and the maximum threshold SHTH as they are, or the minimum threshold SLTH and the maximum threshold SHTH can be scaled by using the difference between these values ​​and the average of n samples.

[0201] For example, the automatic setting unit 96 can automatically generate the gain value SFG. For example, if an actuator can be driven at an intensity in a specific range (0% to 100%), but has been driven at an intensity in a narrower range (30% to 70%) for a certain period of time, the gain value SFG can be adjusted to drive the actuator at an intensity in a wider range.

[0202] For example, increasing the gain value SFG can have substantially the same effect as decreasing the minimum threshold SLTH and the maximum threshold SHTH. Furthermore, decreasing the gain value SFG can have substantially the same effect as increasing the minimum threshold SLTH and the maximum threshold SHTH. In other words, if data is distributed mainly outside the range of the minimum threshold SLTH and the maximum threshold SHTH set by the user, the automatic setting unit 96 can adjust the data outside the range to within the range of the minimum threshold SLTH and the maximum threshold SHTH by adjusting the gain value SFG. This can increase the amount of valid data. However, in such an embodiment, the minimum threshold SLTH and the maximum threshold SHTH preset by the user may be ignored.

[0203] FIG. 26 is a diagram for explaining a user interface according to another embodiment of the present invention.

[0204] The user interface 80a may be generated by a program, which may be executed on a computer, and which may be recorded on a computer-readable recording medium.

[0205] Referring to FIG. 26, a user interface 80 a may include a project management area 81 , an effects management area 82 , an arrangement area 83 , a reference content area 84 , and a timeline area 85 .

[0206] The project management area 81 may include a number of menus for project management. For example, the project management area 81 may include a menu for deleting a project, saving a project, changing a project name, and exporting haptics patterns. The extracted haptic stimulus patterns may be used in the haptic stimulus providing device 10.

[0207] The effect management area 82 may include path effect, dot effect, tactile stimulus frame duplicate, rename effect, and delete effect menus.

[0208] The array region 83 may include a plurality of virtual actuators. Although a plurality of virtual actuators arranged in four rows and five columns is shown in FIG. 2, according to one embodiment, the number and arrangement of the virtual actuators may be changed by manipulating a menu. According to one embodiment, the user may change the arrangement by dragging and dropping each virtual actuator. In this case, instead of a square arrangement, a circular arrangement or an array of another shape may be implemented.

[0209] The reference content area 84 may include reference contents. The reference contents may be video data, sound data, a combination of video and sound data, etc. The reference contents may have time sequence information. The time sequence information of the tactile stimulation pattern generated by the project may correspond to the time sequence information of the reference contents.

[0210] A plurality of tactile stimulation frames l1, l2, l3, l4, r1, r2, and r3 aligned based on time sequence information of the reference content are displayed in the timeline area 85. The size of each tactile stimulation frame may be proportional to the duration for which the effect of the corresponding tactile stimulation frame is provided.

[0211] The timeline region 85 has at least one track, and may have multiple tracks depending on the embodiment. The multiple tracks may correspond to multiple tactile stimulus providing devices, respectively. For example, if a user is expected to wear one tactile stimulus providing device 10 on his left arm and another tactile stimulus providing device 10 on his right arm, the timeline region 85 may be configured to have two tracks. In such a case, a first track 1 may correspond to a tactile stimulus providing device worn on the user's left arm. A second track 2 may correspond to a tactile stimulus providing device worn on the user's right arm.

[0212] Each haptic stimulus frame may be subordinate to a particular track. For example, haptic stimulus frames l1, l2, l3, l4 may be subordinate to a first track (track 1), and haptic stimulus frames r1, r2, r3 may be subordinate to a second track (track 2).

[0213] FIG. 27 is a diagram for explaining the path effect, which is a unit effect.

[0214] 27, the user interface 80a further includes a route effect setting area 86. The route effect setting area 86 can be displayed when the user selects a route effect menu in the effect management area 82. When the route effect menu is selected, a tactile stimulation frame l1 may be generated in the timeline area 85 correspondingly.

[0215] A user can select a path effect menu in the effect management area 82 and draw a path 83p in the arrangement area 83. In one embodiment, a user can draw the path 83p by clicking points 83p1, 83p2, and 83p3 in order using a mouse as an input device. In another embodiment, a user can draw the path 83p at once by dragging using a mouse as an input device. In another embodiment, if the input device is a touchpad, a user can draw the path 83p at once with a finger. A path effect path 1 may be generated in the path effect setting area 86 corresponding to the drawn path 83p. The path effect path 1 is a unit effect, and an applied effect may be generated by combining one or more path effects or by combining a dot effect described later.

[0216] The tactile stimulation pattern generating device 9 can generate a tactile stimulation pattern by setting drive values ​​for a plurality of virtual actuators corresponding to the path 83p on the user interface 80a. The drive values ​​may include elements such as drive strength, drive frequency, drive time, and drive interval.

[0217] According to one embodiment, a tactile stimulation pattern can be generated by setting drive values ​​to virtual actuators that are closest to points corresponding to each time point on the path 83p. For example, a tactile stimulation pattern having a path effect path 1 can be generated by setting drive values ​​to virtual actuators v12, v22, v32, v33, and v34 in that order.

[0218] According to another embodiment, a tactile stimulation pattern can be generated by setting drive values ​​to virtual actuators that form a triangle surrounding points corresponding to each time point of the path 83p. For example, drive values ​​can be set simultaneously to virtual actuators v34, v35, and v44 to provide a tactile stimulation to point 83p3. In this case, the drive values ​​of the virtual actuators that form the triangle may be different from each other. A relatively large drive value can be set to the virtual actuator v34 that is closest to point 83p3, and relatively small drive values ​​can be set to the virtual actuators v35 and v44 that are relatively far from point 83p3.

[0219] Although FIG. 27 shows the tactile stimulation frame l1 having one path effect path 1, the user can also insert additional path effects into the tactile stimulation frame l1 by pressing a path effect add button (plus button) in the path effect setting area 86. Also, the user can delete an already generated path effect from the tactile stimulation frame by pressing a path effect delete button (minus button). When the tactile stimulation frame l1 has multiple path effects, the start and end times of each path effect may be set independently. For example, the start and end times of the first path effect may be set to 0.0 to 0.2 seconds, the second path effect to 0.4 to 0.7 seconds, and the third path effect to 0.6 to 1.0 seconds.

[0220] A user can adjust the drive strength of a plurality of virtual actuators for which drive values ​​are set by adjusting a power value in the path effect setting area 86. The size of the graphic symbol displayed at the points 83p1, 83p2, and 83p3 may change according to the power value. For example, if the power value is high, the graphic symbol may be displayed large, and if the power value is low, the graphic symbol may be displayed small. Thus, a user can intuitively recognize the magnitude of the corresponding vibration force according to the size of the graphic symbol at each point 83p1, 83p2, and 83p3.

[0221] The user can adjust the time length of the path effect path1 by adjusting the duration value in the path effect setting area 86. The user can also adjust the time length of the path effect 1 by dragging the side of the corresponding tactile stimulation frame l1 in the timeline area 85.

[0222] A user can also adjust a playing pattern in the path effect setting area 86. In one embodiment, when a constant speed is selected as the playing pattern, drive values ​​may be set for the virtual actuators so that the haptic stimulus is applied at a constant speed from the start point 83p1 to the end point 83p3. In another embodiment, when a constant time to each point is selected as the playing pattern, drive values ​​may be set for the virtual actuators so that the haptic stimulus is applied at a slow speed from the point 83p1 to the point 83p2 and at a fast speed from the point 83p2 to the point 83p3.

[0223] A user can adjust the texture of the path effect Path 1 in the path effect setting area 86. The closer the texture is to "soft", the closer the connections between the actuators are to each other and the actuators are driven smoothly, whereas the closer the texture is to "rough", the closer the connections between the actuators are to each other and the actuators are driven choppily.

[0224] A user can adjust the dynamic settings in the path effect settings area 86. Dynamic settings may include fade in, fade out, no setting, etc.

[0225] FIG. 28 is a diagram for explaining the dot effect which is a unit effect.

[0226] 28, the user interface 80a further includes a dot effect setting area 87. The dot effect setting area 87 may be displayed when a user selects a dot effect menu in the effect management area 82. When the dot effect menu is selected, a tactile stimulation frame l3 may be generated in the timeline area 85.

[0227] A user can select a dot effect menu in the effect management area 82 and select at least one virtual actuator v22, v31, v32, v33, v42 in the arrangement area 83. In one embodiment, the user can use a mouse as an input device to click on the actuators v22, v31, v32, v33, v42, respectively. In response to the user's input, a dot effect dot1 may be generated in the dot effect setting area 87. The dot effects dot1 and dot2 are unit effects, and an applied effect may be generated by combining one or more dot effects or by combining with the above-mentioned path effect.

[0228] FIG. 28 shows that the tactile stimulation frame l3 has two dot effects, dot1 and dot2. The user can also insert an additional dot effect into the tactile stimulation frame l3 by pressing the dot effect addition button (plus button) in the dot effect setting area 87. The user can also delete the already generated dot effects dot1 and dot2 from the tactile stimulation frame l3 by pressing the dot effect deletion button (minus button). When the tactile stimulation frame l3 has multiple dot effects, the start and end times of each dot effect may be set independently. For example, the start and end times of the first dot effect may be set to 0.0 to 0.2 seconds, the second dot effect to 0.4 to 0.7 seconds, and the third dot effect to 0.6 to 1.0 seconds.

[0229] The user can adjust the drive strength of the virtual actuators v22, v31, v32, v33, and v42, the drive values ​​of which are set, by adjusting the power values ​​in the dot effect setting area 87. The size of the graphic symbol displayed on the virtual actuators v22, v31, v32, v33, and v42 may change according to the power values. For example, if the power value is high, the graphic symbol may be displayed large, and if the power value is low, the graphic symbol may be displayed small. Thus, the user can intuitively recognize the magnitude of the corresponding vibration force according to the size of the graphic symbol of each virtual actuator v22, v31, v32, v33, and v42.

[0230] The user can also adjust the duration of the dot effects dot1, dot2 by adjusting the duration value in the dot effect setting area 87. The user can also adjust the duration of the dot effects by dragging the side of the corresponding tactile stimulation frame l3 in the timeline area 85.

[0231] A user can adjust the playback pattern in the dot effect setting area 87. If the playback pattern is discrete, the first dot effect dot1 and the second dot effect dot2 can have separate drive values ​​with no connectivity to the respective virtual actuators. If the playback pattern is connected dots, the first dot effect dot1 and the second dot effect dot2 can have temporal connectivity as if they were one effect, and the virtual actuators can have temporally connected drive values.

[0232] The user can adjust the texture in the dot effect setting area 87. The texture is meaningful when the playback pattern is connected, and can make the connectivity between the first dot effect dot1 and the second dot effect dot2 smooth (soft) or somewhat rough (rough). When the texture is set to be rough, a tactile stimulus that feels close to a discrete type can be provided even if the playback pattern is connected.

[0233] FIG. 29 is a diagram for explaining the arrangement of tactile stimulation frames according to one embodiment of the present invention.

[0234] As described above, the tactile stimulation frames l1 and l3 may be frames created directly by the user using a mouse or the like. However, the tactile stimulation frame l4 may not be a frame created by the user, but may be a frame automatically generated by the tactile stimulation pattern generating device 9 based on the first audio data. For example, when a gunshot is heard from the right side, a driving strength value may not be given to the virtual actuator in the tactile stimulation frame l4 of the first track track1 linked to the tactile stimulation providing device 10 worn on the user's left arm. For example, the driving strength values ​​of the virtual actuators in the tactile stimulation frame l4 may all be 0.

[0235] According to the present embodiment, the timeline region 85 may include a first haptic stimulus frame based on the first audio data and a second haptic stimulus frame based on the second audio data, where the first haptic stimulus frame may be subordinated to a first track and the second haptic stimulus frame may be subordinated to a second track.

[0236] The drawings and detailed description of the invention described above are merely illustrative of the present invention and are used only to explain the present invention, and are not used to limit the meaning or the scope of the present invention described in the claims. Therefore, a person having ordinary skill in the art will understand that various modifications and equivalent other embodiments are possible. Therefore, the true technical scope of protection of the present invention should be determined by the technical ideas of the accompanying claims.

Claims

1. a tactile stimulation pattern generating unit that generates a first actuator strength value corresponding to a representative value of n frequency bands of the first audio data and a second actuator strength value corresponding to a representative value of m frequency bands of the second audio data; a tactile stimulus providing unit including p actuators and driving the p actuators based on the n first actuator strength values ​​and the m second actuator strength values; where n, m, and p are integers greater than 1; the first audio data includes information relating to a first direction; the second audio data includes information regarding a second direction different from the first direction; The tactile stimulation pattern generating unit is a data converter for converting the first audio data and the second audio data in a time domain into first converted data and second converted data in a frequency domain, respectively; a band representative value calculation unit that calculates the representative value of the frequency band for each of the first converted data and the second converted data, A tactile stimulation providing device, characterized in that the representative value is based on valid data that is greater than a minimum threshold value for each of the frequency bands.

2. The tactile stimulation pattern generating unit is The tactile stimulus providing device according to claim 1 , further comprising an actuator strength calculation unit that calculates an actuator strength value corresponding to the representative value.

3. The tactile stimulation pattern generating unit is The tactile stimulus providing device according to claim 2 , further comprising an actuator strength attenuation unit that gradually attenuates the actuator strength value when the actuator strength value is maintained for a certain period of time or more.

4. The tactile stimulation pattern generating unit is The tactile stimulus providing device according to claim 3 , further comprising an actuator strength merging unit that generates one actuator strength value by merging two or more actuator strength values ​​mapped to one actuator.

5. The tactile stimulation pattern generating unit is The tactile stimulus providing device according to claim 4 , further comprising an automatic setting unit for setting the minimum threshold value based on a previous representative value of each of the frequency bands.

6. The tactile stimulation providing device according to claim 5 , wherein the minimum threshold is set based on a confidence interval algorithm or a prediction interval algorithm.

7. The tactile stimulation providing device according to claim 5 , wherein the minimum threshold is set based on an estimate of a non-parametric prediction interval.

8. The tactile stimulus providing device according to claim 1, characterized in that p is equal to or smaller than the sum of n and m.

9. an actuator corresponding to the first position information corresponding to the first direction is driven based on at least one of the first actuator strength values; The tactile stimulus providing device according to claim 1 , wherein an actuator corresponding to second position information corresponding to the second direction is driven based on at least one of the second actuator strength values.

10. The tactile stimulation providing device according to claim 1 , wherein the representative value is based on valid data that is greater than a minimum threshold and less than a maximum threshold for each of the frequency bands.

11. The first audio data corresponds to data of a first channel of stereo audio data, 2. The tactile stimulus providing device according to claim 1, wherein the second audio data corresponds to data of a second channel of stereo audio data.

12. The first audio data corresponds to data of a first channel of surround audio data having three or more channels, The tactile stimulus providing device according to claim 1 , wherein the second audio data corresponds to second channel data of the surround audio data.

13. The tactile stimulus providing device of claim 1, wherein at least one of the p actuators is driven based on at least one of the n first actuator strength values ​​and at least one of the m second actuator strength values.

14. The tactile stimulus providing device according to claim 1 , wherein at least one of the p actuators is driven based on at least two of the n first actuator strength values.

15. The tactile stimulus providing device according to claim 1 , wherein at least two of the frequency bands overlap at least partially.

16. The tactile stimulation providing device of claim 2, characterized in that the actuator strength calculation unit calculates the actuator strength value so that the smaller the representative value is in a section where the representative value is smaller than a reference representative value corresponding to the maximum value of the actuator strength value, and calculates the actuator strength value so that the larger the representative value is in a section where the representative value is larger than the reference representative value, the smaller the actuator strength value is.

17. The tactile stimulus providing device according to claim 2 , wherein the actuator strength calculation unit determines that the relationship between the representative value and the actuator strength value includes a discontinuous function.

18. The tactile stimulus providing device according to claim 1 , wherein a first minimum threshold value corresponding to a first frequency band among the frequency bands is different from a second minimum threshold value corresponding to a second frequency band among the frequency bands.

19. a tactile stimulation pattern generating unit that generates a first actuator strength value corresponding to a representative value of n frequency bands of the first audio data and a second actuator strength value corresponding to a representative value of m frequency bands of the second audio data; a tactile stimulus providing unit including p actuators and driving the p actuators based on the n first actuator strength values ​​and the m second actuator strength values; where n, m, and p are integers greater than 1; the first audio data includes information relating to a first direction; the second audio data includes information regarding a second direction different from the first direction; the representative values ​​are based on valid data greater than a minimum threshold value for each of the frequency bands; The tactile stimulation pattern generating unit is The tactile stimulus providing device further comprises an automatic setting unit that sets the minimum threshold value based on a previous representative value of each of the frequency bands.

20. The tactile stimulus providing device according to claim 19 , wherein the automatic setting unit sets a maximum threshold value for each of the frequency bands based on the previous representative value for each of the frequency bands.

21. The tactile stimulus providing device according to claim 20 , wherein the automatic setting unit sets the gain value of each of the frequency bands based on the strength of the actuator in a previous period.

22. The tactile stimulation providing device of claim 20, wherein the automatic setting unit sets the minimum threshold and the maximum threshold for each of the frequency bands so that the minimum threshold and the maximum threshold are limited within a range between a user minimum threshold and a user maximum threshold.

23. The tactile stimulus providing device according to claim 22 , wherein the automatic setting unit sets a gain value for each of the frequency bands in order to adjust data outside the range to within the range.

Images