Method and apparatus for generating haptic signals using audio signal patterns

Abstract

The present invention relates to a haptic signal generating method, characterized by including the steps of analyzing a received audio signal and generating a signal pattern composed of signal magnitudes for each time and frequency band, comparing a target audio signal pattern with the generated signal pattern, and generating a haptic signal corresponding to the target audio signal pattern if the matching rate exceeds a threshold as a result of the comparison.If a specific audio signal that is to be output as a haptic signal exists among various audio signals, the specific audio signal pattern can be detected and an appropriate haptic signal can be connected to the output.

Description

[Technical Field]

[0001] The present invention relates to a method for generating a haptic signal, and more particularly to a method and apparatus for generating a haptic signal using an audio signal pattern, which can detect a specific audio signal pattern when a specific audio signal that is to be output as a haptic signal is present among various audio signals, and connect an appropriate haptic signal to the output. [Background technology]

[0002] Products that support haptic technology enable greater realism and immersion by adding tactile information to users. However, implementing haptic technology using content that does not already contain haptic information requires a significant amount of time, effort, and expense because vibration patterns must be embedded in the content in advance to match specific times and purposes when the content is played. To address this issue, there is a need for haptic technology that automatically converts existing audio signals into haptic signals.

[0003] In the most typical existing technology, when converting an audio signal into a haptic signal, the audio signal is analyzed in the time domain or frequency domain, and a filter is used to extract signals concentrated in a specific region and connect them to the haptic signal. However, the method of generating a haptic signal using the time and frequency patterns of an audio signal has not been used in the prior art, and instead, the method of using the results of analyzing the audio signal in the time domain or frequency domain has been used.

[0004] Prior Art 1 (Korean Patent Publication No. 10-2011-0076283) is a technology relating to a method and device for providing feedback according to a user's input pattern, where the input audio pattern consists of the type of sound source generated in response to the user's input pattern, the duration of the generation, the volume, etc.

[0005] Prior Art 2 (Korean Patent Publication No. 10-2012-0126446) is a technology related to a device for generating vibration feedback from an input audio signal, which derives parameters including at least one of vibration strength and vibration roughness from the audio signal to generate a haptic signal.

[0006] Prior Art 1 or Prior Art 2 generates a haptic signal using a method that is completely different from the method of generating a haptic signal using the time and frequency patterns of an audio signal. Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, the first problem that the present invention aims to solve is to provide a haptic signal generating method that can detect a specific audio signal pattern and connect an appropriate haptic signal to the output when a specific audio signal that is to be output as a haptic signal exists among various audio signals.

[0008] The second problem that the present invention aims to solve is to provide a haptic signal generating device that can have a high accuracy rate because the output haptic signal is matched to a pre-specified audio signal pattern, and can fundamentally block the generation of unwanted haptic signals such as background sounds and human voices.

[0009] Another object of the present invention is to provide a computer-readable recording medium on which a program for executing the method by a computer is recorded. [Means for solving the problem]

[0010] To achieve the first object, the present invention provides a haptic signal generating method including the steps of: analyzing a received audio signal and generating a signal pattern composed of signal magnitudes for each time and frequency band; comparing a target audio signal pattern with the generated signal pattern; and, if a matching rate exceeds a threshold as a result of the comparison, generating a haptic signal corresponding to the target audio signal pattern.

[0011] According to an embodiment of the present invention, the method may further include scaling the generated signal pattern in consideration of a difference in magnitude between feature points of the generated signal pattern and feature points of the target audio signal pattern.

[0012] Also, feature points of the target audio signal pattern and the generated signal pattern may be extracted, and a matching rate may be calculated based on the difference between the feature points.

[0013] To achieve the second object, the present invention provides a haptic signal generating device including a signal pattern generating unit that analyzes a received audio signal and generates a signal pattern composed of signal magnitudes for each time and frequency band, a pattern comparing unit that compares a target audio signal pattern with the generated signal pattern, and a haptic signal generating unit that generates a haptic signal corresponding to the target audio signal pattern if a matching rate exceeds a threshold as a result of the comparison.

[0014] According to an embodiment of the present invention, the audio signal processing device may further include a scaling unit that scales the generated signal pattern in consideration of a difference in magnitude between feature points of the generated signal pattern and feature points of the target audio signal pattern.

[0015] Preferably, the pattern comparison unit extracts feature points of the target audio signal pattern and the generated signal pattern, and calculates a matching rate based on a difference between the feature points.

[0016] In order to solve the other technical problems, the present invention provides a computer-readable recording medium having a program recorded thereon for causing a computer to execute the haptic signal generating method. [Effects of the Invention]

[0017] According to the present invention, if there is a specific audio signal among various audio signals that you want to output as a haptic signal, you can detect the specific audio signal pattern and connect the appropriate haptic signal to the output.

[0018] In addition, according to the present invention, since the output haptic signal is matched to a pre-specified audio signal pattern, it is possible to have a high accuracy rate and fundamentally block the generation of unwanted haptic signals such as background sounds and human voices.

[0019] Furthermore, according to the present invention, feature points are extracted from the signal pattern and the matching rate is compared, allowing for quick comparison. Even in the presence of noise, if the feature points of the signal pattern match, a haptic signal is generated, thereby ensuring a uniform haptic output even in diverse and noisy situations. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a configuration diagram of a haptic signal generating device according to a preferred embodiment of the present invention. [Figure 2] 1 shows the waveform of an audio signal as a function of time. [Figure 3] 3 illustrates a signal pattern obtained by analyzing the audio signal of FIG. 2 input according to an embodiment of the present invention by time and frequency band. [Figure 4] 1 is a flowchart of a haptic signal generating method according to a preferred embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] To achieve the above object, the present invention provides a haptic signal generating method including the steps of: analyzing a received audio signal and generating a signal pattern composed of signal magnitudes for each time and frequency band; comparing a target audio signal pattern with the generated signal pattern; and, if a matching rate exceeds a threshold as a result of the comparison, generating a haptic signal corresponding to the target audio signal pattern.

[0022] Hereinafter, preferred embodiments that can be easily implemented by those skilled in the art will be described in detail with reference to the accompanying drawings. However, it will be apparent to those skilled in the art that these embodiments are merely for the purpose of explaining the present invention in more detail and that the scope of the present invention is not limited thereto.

[0023] The configuration of the present invention will be described in detail with reference to the accompanying drawings based on preferred embodiments of the present invention in order to clarify the solution to the problem to be solved by the present invention. When reference numerals are used to designate components in the drawings, the same reference numerals are used to designate the same components in other drawings, and it is to be made clear that components in other drawings can be referenced when necessary when describing the drawings. Furthermore, in describing the operating principles of the preferred embodiments of the present invention in detail, detailed descriptions of well-known functions or configurations related to the present invention and other matters will be omitted if they are deemed to unnecessarily obscure the gist of the present invention.

[0024] FIG. 1 is a diagram showing the configuration of a haptic signal generating device according to a preferred embodiment of the present invention.

[0025] Referring to FIG. 1, the haptic signal generating device according to the present embodiment includes a signal pattern generating unit 110, a target pattern storing unit 120, a scaling unit 130, a pattern comparing unit 140, a haptic signal generating unit 150, and a motor driving unit 160.

[0026] The signal pattern generator 110 analyzes a received audio signal and generates a signal pattern consisting of signal amplitudes for each time and frequency band.

[0027] The signal pattern generator 110 can analyze the audio signal in the time domain or by converting it into the frequency domain. When converting it into the frequency domain, it is preferable to convert the audio signal into each frequency band using a Fast Fourier Transform (FFT).

[0028] The target pattern storage unit 120 stores an audio signal pattern for generating a haptic signal as a target audio signal pattern, and can also store a haptic signal corresponding to the stored target audio signal pattern.

[0029] Figure 2 shows the waveform of an audio signal over time.

[0030] FIG. 3 shows a signal pattern obtained by analyzing the input audio signal of FIG. 2 according to an embodiment of the present invention by time and frequency band.

[0031] Referring to FIG. 3, the horizontal axis represents time, the vertical axis represents frequency, and the intersection of time and frequency represents the magnitude of the signal.

[0032] Blue indicates that the signal magnitude at that frequency is small at that time, and red indicates that the signal magnitude is large.

[0033] As shown in Figure 3, a unique audio signal pattern can be generated by analyzing an audio signal by time and frequency band, and when the degree to which the generated signal pattern matches a target audio signal pattern desired by the user is analyzed, if a matching rate above a threshold is obtained, a corresponding haptic signal is generated. Since the signal pattern generated from the audio signal is compared with a pre-stored signal pattern, a fairly high level of accuracy can be achieved depending on the matching rate.

[0034] The scaling unit 130 scales the generated signal pattern in consideration of the difference in magnitude between the feature points of the signal pattern generated by the signal pattern generating unit 110 and the feature points of the target audio signal pattern stored in the target pattern storage unit 120.

[0035] The pattern comparison unit 140 may extract feature points of the target audio signal pattern and the generated signal pattern and calculate a matching rate based on the difference between the feature points. When feature points of two signal patterns are extracted and the matching rate is calculated based on the difference between corresponding feature points, the time required to calculate the matching rate can be reduced compared to when the entire signal patterns are compared.

[0036] Referring again to FIG. 2, the pattern comparison unit 140 can compare the signal patterns shown in FIG. 2 with each other for a specific frequency band during a specific time period.

[0037] If the pattern comparison unit 140 determines, as a result of comparing the signal patterns, that a specific frequency band within a specific time period of the signal pattern is a region of the signal pattern that has characteristics that distinguish the target audio signal from other audio signals, it is preferable to reduce the time and frequency band of the target audio signal pattern to the specific time and specific frequency band.

[0038] If the matching rate exceeds a threshold as a result of the comparison by the pattern comparator 140, the haptic signal generator 150 generates a haptic signal corresponding to the target audio signal pattern.

[0039] On the other hand, if the number of occurrences of haptic signals is counted and a haptic signal corresponding to one target audio signal pattern occurs excessively more than a certain number of times, it is highly likely that a matching error has occurred, so it is preferable to increase the matching rate threshold.

[0040] The motor driver 160 receives the haptic signal from the haptic signal generator 150 and drives the haptic motor.

[0041] FIG. 4 is a flowchart of a haptic signal generating method according to a preferred embodiment of the present invention.

[0042] 4, the haptic signal generating method according to this embodiment is composed of steps that are processed in time sequence in the haptic signal generating device shown in FIG 1. Therefore, even if some content is omitted below, the content described above regarding the haptic signal generating device shown in FIG 1 also applies to the haptic signal generating method according to this embodiment.

[0043] In step 410, the haptic signal generator analyzes the received audio signal and generates a signal pattern composed of signal amplitudes for each time and frequency band.

[0044] The received audio signal can be analyzed in the time domain or converted to the frequency domain for analysis. When converting to the frequency domain, the audio signal can be converted into each frequency band using a Fast Fourier Transform (FFT).

[0045] In step (420), the haptic signal generating device scales the generated signal pattern, taking into account the difference in magnitude between the feature points of the signal pattern generated in step (410) and the feature points of the target audio signal pattern stored in the target pattern storage unit 120.

[0046] In step 430, the haptic signal generator compares the target audio signal pattern with the generated signal pattern.

[0047] The signal patterns can be compared by extracting feature points from the target audio signal pattern and the generated signal pattern and calculating a matching rate based on the difference between the feature points. When feature points of two signal patterns are extracted and the matching rate is calculated based on the difference between corresponding feature points, the time required to calculate the matching rate can be reduced compared to the prior art when the entire signal patterns are compared.

[0048] On the other hand, if, as a result of comparing the signal patterns, it is determined that a specific frequency band during a specific time period of the signal pattern is a region of the signal pattern that has characteristics that distinguish the target audio signal from other audio signals, it is preferable to reduce the time and frequency band of the target audio signal pattern to the specific time and specific frequency band.

[0049] In step 440, if the matching rate exceeds the threshold, the haptic signal generator proceeds to step 450, and if the matching rate does not exceed the threshold, the haptic signal generator ends the process.

[0050] In step 450, if the matching rate exceeds a threshold as a result of the determination in step 440, the haptic signal generator generates a haptic signal corresponding to the target audio signal pattern.

[0051] On the other hand, if the number of occurrences of a haptic signal is counted and the haptic signal corresponding to the target audio signal pattern occurs excessively more than a certain number of times, it is highly likely that a matching error has occurred, so it is preferable to increase the threshold of the matching rate.

[0052] A haptic signal corresponding to the target audio signal pattern may be stored in the target pattern storage unit 120 and then transmitted to the haptic signal generator 150 .

[0053] Embodiments of the present invention may be embodied in the form of program instructions executed by various computer means and stored on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions stored on the medium may be those specially designed and constructed for the present invention, or they may be well known and available to those skilled in the art of computer software. Examples of computer-readable storage media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as produced by a compiler, but also high-level language code executed by a computer using an interpreter, for example. The hardware devices may also be configured to operate as one or more software modules to perform the operations of the present invention, or vice versa.

[0054] The present invention has been described above using specific details such as specific components and limited embodiments and drawings, but this is merely provided for a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Various modifications and variations can be made from such descriptions by those having ordinary knowledge in the technical field to which the present invention pertains.

[0055] Therefore, the concept of the present invention should not be limited to the described embodiments, and it can be said that not only the scope of the claims described below, but also anything that has equivalent or equivalent modifications to the scope of these claims falls within the scope of the concept of the present invention.

Claims

1. Analyzing the received audio signal and generating a signal pattern composed of signal magnitudes for each time and frequency band; comparing a target audio signal pattern with the generated signal pattern; generating a haptic signal corresponding to the target audio signal pattern when a matching rate exceeds a threshold as a result of the comparison; Including, A haptic signal generating method, wherein the comparing step reduces the time and frequency band to be compared with the target audio signal pattern to the specific time and specific frequency band if it is determined that a specific frequency band for a specific time of the signal pattern is a region of the signal pattern having characteristics corresponding to the target audio signal.

2. The method further comprises the step of scaling the generated signal pattern in consideration of a difference in magnitude between feature points of the generated signal pattern and feature points of the target audio signal pattern. The haptic signal generating method of claim 1 .

3. extracting feature points from the target audio signal pattern and the generated signal pattern, and calculating a matching rate based on a difference between the feature points. The haptic signal generating method of claim 1 .

4. a signal pattern generator that analyzes a received audio signal and generates a signal pattern consisting of signal amplitudes for each time and frequency band; a pattern comparison unit that compares a target audio signal pattern with the generated signal pattern; a haptic signal generator that generates a haptic signal corresponding to the target audio signal pattern when the matching rate exceeds a threshold as a result of the comparison; Including, A haptic signal generating device in which, if the pattern comparison unit determines that a specific frequency band during a specific time of the signal pattern is a region of the signal pattern having characteristics corresponding to the target audio signal, the pattern comparison unit reduces the time and frequency band to be compared with the target audio signal pattern to the specific time and specific frequency band.

5. The method further comprises: scaling the generated signal pattern in consideration of a difference in magnitude between feature points of the generated signal pattern and feature points of the target audio signal pattern. The haptic signal generating device according to claim 4 .

6. The pattern comparison unit extracts feature points of the target audio signal pattern and the generated signal pattern, and calculates a matching rate based on a difference between the feature points. The haptic signal generating device according to claim 4 .

7. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the method according to any one of claims 1 to 3.

Images