Tactile sense transmission system, tactile sense transmission device, tactile sense transmission program, and tactile sense transmission method

The tactile transmission system addresses howling and loopback issues by filtering and converting tactile signals to frequencies distinct from the housing's natural vibration, ensuring clear tactile communication in two-way systems.

JP2024166245AActive Publication Date: 2024-11-28TOHOKU UNIV
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
JP2024153096
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-28
Estimated Expiration
2041-06-04

AI Technical Summary

Technical Problem

The issue of howling and loopback of touch signals occurs in two-way tactile transmission systems due to the bidirectional connection of vibration sensors and actuators, which are driven around the same natural vibration frequency, leading to interference and a decrease in tactile information quality.

Method used

A tactile transmission system that includes a vibration measurement unit, a calculation unit to determine perceptual information, and a conversion unit that filters and converts the signal to a predetermined frequency to suppress loopback and howling by attenuating specific frequency bands.

Benefits of technology

The system effectively suppresses howling and loopback, maintaining tactile signal quality by converting signals to frequencies different from the natural vibration frequency of the housing, thereby preventing interference and ensuring clear tactile communication.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024166245000001_ABST
    Figure 2024166245000001_ABST
Patent Text Reader

Abstract

To prevent howling and loop back of a contact signal in a bi-directional tactile sense transmission system.SOLUTION: A tactile sense transmission system 100 has a tactile sense transmission device 3 and an other tactile sense transmission device 3, and comprises: a vibration measuring unit 31 that measures vibration occurring in the tactile sense transmission device 3; a calculation unit that calculates perceptual information specified from the vibration measured by the vibration measuring unit 31; a conversion unit that converts a signal related to the vibration into a predetermined frequency while maintaining the perceptual information calculated by the calculation unit; and a signal output unit that causes an oscillator 32 of the other tactile sense transmission device 3 to output the signal after the conversion by the conversion unit as output vibration.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The technology described in this specification relates to a tactile transmission system, a tactile transmission device, a tactile transmission program, and a tactile transmission method. [Background technology]

[0002] Technologies are being developed to communicate with voice and images even in remote locations, such as telephone, video conferencing, and social virtual reality (VR). By transmitting haptic vibrations in addition to voice and images, more advanced communication is possible than ever before. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Hirohiko Hayakawa et al. “Proposal of a bidirectional audiovisual haptic media for remote communication with a high sense of reality: Public haptic legends”, TVRSJ Vol.25 No.4 pp.412-421, December 25, 2020 Summary of the Invention [Problem to be solved by the invention]

[0004] In haptic communication, which transmits vibrations generated when a user touches a housing in both directions, it is desirable to mount a vibration sensor that measures vibrations and an actuator that generates vibrations on the same housing. In this case, the contact vibration between the user and the housing that is applied as input appears as an amplitude modulated wave of the natural vibration of the housing. On the other hand, when the actuator vibrates the housing to present a haptic sensation as output, vibrations close to the natural vibration of the housing are also generated.

[0005] When multiple such tactile input / output devices are connected in both directions, the sensor signal and the looped-back actuator drive signal are driven near the same natural vibration, which may result in a ringing noise (howling).

[0006] In order to suppress this feedback, a bandpass filter or other means is used to suppress signals in the natural vibration frequency band for acoustic signals. However, in the case of tactile transmission, the tactile information to be conveyed also exists in the same band, so lowering the gain may result in a deterioration in the quality of the tactile information.

[0007] In one aspect, the technology described herein aims to suppress feedback and loopback of a touch signal in a bidirectional tactile transmission system. [Means for solving the problem]

[0008] In one aspect, a bidirectional tactile transmission system is a tactile transmission system having a tactile transmission device and another tactile transmission device, and includes a vibration measuring unit that measures vibrations generated in the tactile transmission device, a calculation unit that calculates perceptual information identified from the vibrations measured by the vibration measuring unit, a conversion unit that converts a signal related to the vibration to a predetermined frequency while maintaining the perceptual information calculated by the calculation unit, and a signal output unit that outputs the converted signal converted by the conversion unit as an output vibration to a vibrator of the other tactile transmission device, and the conversion unit attenuates a specific frequency band by filtering so as to suppress loopback of the vibration output from the vibrator. In another aspect, a bidirectional tactile transmission system is a tactile transmission system having a tactile transmission device and another tactile transmission device, and includes a vibration measuring unit that measures vibrations generated in the tactile transmission device, a calculation unit that calculates perceptual information identified from the vibrations measured by the vibration measuring unit, a conversion unit that converts a signal related to the vibration to a predetermined frequency while maintaining the perceptual information calculated by the calculation unit, and a signal output unit that outputs the converted signal converted by the conversion unit as an output vibration to a vibrator of the other tactile transmission device, and the conversion unit converts the signal to a frequency other than the resonant frequency of a housing of the tactile transmission device as the predetermined frequency. Effect of the Invention

[0009] As one aspect, it is possible to suppress feedback and loopback of a touch signal in a bidirectional tactile transmission system. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating an example of a hardware configuration of a bidirectional tactile sense transmission system according to an embodiment. [Diagram 2] 2A and 2B are block diagrams illustrating an example of the software configuration of the bidirectional tactile transmission system illustrated in FIG. 1. [Diagram 3] FIG. 3(a) is a block diagram showing a first example of a signal input processing unit shown in FIG. 2, and FIG. 3(b) is a block diagram showing a first example of a signal output processing unit shown in FIG. [Figure 4] 3(a) is a block diagram showing a second example of the signal input processing unit shown in FIG. 2, and (b) is a block diagram showing a second example of the signal output processing unit shown in FIG. [Diagram 5] 3(a) is a block diagram showing a third example of the signal input processing unit shown in FIG. 2, and (b) is a block diagram showing a third example of the signal output processing unit shown in FIG. [Figure 6] 5(a) to 5(c) are graphs for explaining the frequency conversion process performed by the frequency conversion unit shown in FIG. 3 and FIG. [Figure 7] 11 is a graph illustrating the measurement results of the amplitude at a through output (when a signal is output without processing). [Figure 8] 2 is a table illustrating the results of suppressing feedback when Intensity Segment Modulation (ISM) is used in the bidirectional tactile transmission system shown in FIG. 1 . [Figure 9] 13A is a graph illustrating the measurement results of output vibration in the case of through output, and FIG. 13B is a graph illustrating the measurement results of output vibration in the case of using ISM. [Figure 10] FIG. 1 is a diagram illustrating a first example of an environmentally-installed vibration input / output device. [Figure 11]FIG. 11 is a diagram illustrating a second example of an environmentally-installed vibration input / output device. [Figure 12] FIG. 11 is a diagram illustrating a third example of an environmentally-installed vibration input / output device. [Figure 13] FIG. 1 is a diagram showing a wristband-type vibration input / output device. [Figure 14] FIG. 1 is a diagram showing a pillow-type vibration input / output device. [Figure 15] FIG. 1 is a diagram showing a tool-mounted vibration input / output device. [Figure 16] FIG. 2 is a block diagram illustrating a configuration example of a terminal according to an embodiment. [Figure 17] Graphs (a) to (c) briefly explain the ISM process. [Figure 18] 1 is a graph showing the discriminability of vibration by humans. [Figure 19] 19 is a sample waveform of vibration used in a forced three-choice discrimination experiment conducted to determine the discriminability shown in the graph shown in FIG. 18. [Figure 20] 17 is a graph showing signal waveforms before and after conversion for each segment by the terminal shown in FIG. 16. [Figure 21] 13 is a graph showing an amplitude threshold Tf used in calculating the correction energy. [Figure 22] 13 is a graph showing the exponent threshold b used in calculating the corrected energy. [Figure 23] 17 is a diagram for explaining the use of a window function in the terminal shown in FIG. 16. [Figure 24] 17 is a graph illustrating an example of combining low and high frequencies in the terminal shown in FIG. 16. [Diagram 25] 17 is a graph showing a specific example of signal waveforms before and after conversion by the terminal shown in FIG. 16. [Figure 26] 17 is a block diagram illustrating an example of a functional configuration of an ISM unit in the terminal illustrated in FIG. 16. [Figure 27] 17 is a block diagram for explaining a first embodiment of a vibration waveform generation process in the terminal shown in FIG. 16. [Figure 28]FIG. 27 is a block diagram illustrating details of the energy control process shown in FIG. 26. [Figure 29] 27 is a block diagram for explaining a process of separating low-frequency components in the energy control process shown in FIG. 26 as a second example of a process of generating a vibration waveform in the terminal shown in FIG. [Diagram 30] 13(a) to 13(c) are graphs illustrating an example in which vibration is generated according to ISM without enhancing the waveform. [Diagram 31] 13(a) to 13(c) are graphs illustrating a first example of emphasizing and separating high frequency components of 3000 Hz or higher from a sound source. [Diagram 32] 6(a) to 6(c) are graphs illustrating a second example of emphasizing and separating high frequency components of 3000 Hz or higher from a sound source. [Diagram 33] 13(a) to 13(c) are graphs illustrating an example of emphasizing and separating low-frequency components of 1000 Hz or less from a sound source. [Diagram 34] FIG. 27 is a block diagram illustrating a first modified example of the energy control process shown in FIG. 26. [Diagram 35] FIG. 27 is a block diagram illustrating a second modified example of the energy control process shown in FIG. [Diagram 36] FIG. 27 is a block diagram illustrating details of the energy synthesis process shown in FIG. 26. [Figure 37] 27 is a block diagram illustrating the details of a generation process of a corrected vibration waveform shown in FIG. 26. [Figure 38] 2 is a block diagram showing an example of the configuration of a DAC when a plurality of vibration devices are used in the bidirectional tactile transmission system shown in FIG. 1. [Figure 39] 2 is a block diagram showing an example of the configuration of a DAC when a single vibration device is used in the bidirectional tactile transmission system shown in FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment will be described with reference to the drawings. However, the embodiment described below is merely an example, and there is no intention to exclude various modifications and application of techniques not clearly stated in the embodiment. In other words, the present embodiment can be implemented with various modifications within the scope of the gist of the present invention.

[0012] In addition, each figure does not necessarily include only the components shown in the figure, but may include other components. In the following drawings, parts with the same reference numerals indicate the same or similar parts unless otherwise specified.

[0013] [A] Embodiment [A-1] Two-way tactile communication processing

[0014] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a bidirectional tactile sense transmission system 100 according to an embodiment.

[0015] The bidirectional tactile transmission system 100 illustratively includes two terminals 1 (in other words, tactile transmission devices), two Universal Serial Bus (USB) audio interfaces (I / F) 2, two vibration input / output devices 3 (vibration input / output devices #1 and #2), and two amplifiers 4. Each of the two vibration input / output devices 3 includes a vibration sensor 31 (in other words, a vibration measuring unit) and a vibrator 32 (in other words, a vibrator).

[0016] In FIG. 1, solid arrows indicate the signal flow when vibrations generated in vibration input / output device #2 are transmitted to vibration input / output device #1, and dashed arrows indicate the signal flow when vibrations generated in vibration input / output device #1 are transmitted to vibration input / output device #2.

[0017] The vibration sensor 31 measures vibration and transmits a signal to the terminal 1 via the USB audio I / F 2. The terminal 1 executes ISM processing or through output processing to transmit a signal to another terminal 1 at a remote location. The remote terminal 1 outputs vibration generated from the transmitted signal to the vibrator 32 via the USB audio I / F 2 and the amplifier 4. Note that three or more sets of terminals 1, USB audio I / F 2, vibration input / output device 3, and amplifier 4 may be provided, so that tactile sensation may be transmitted between three or more parties. Also, for users who do not want to experience the transmission of tactile sensation, a mute function for turning off the output of vibration by the vibrator 32 may be provided.

[0018] The ISM will be described in detail later with reference to FIGS.

[0019] 2(a) and 2(b) are block diagrams illustrating an example of the software configuration of the bidirectional tactile transmission system 100 shown in FIG.

[0020] As shown in (a) and (b) of Figures 2, the bidirectional tactile transmission system 100 functions as a vibration measurement unit 211, a signal amplification unit 212, a signal input processing unit 213, a signal transmission unit 214, a signal receiving unit 215, a signal output processing unit 216, a signal amplification unit 217, and a vibration presentation unit 218 at each of locations #1 and #2.

[0021] The functions of the vibration measuring unit 211 and the signal amplifying unit 212 may be realized by the vibration sensor 31 shown in Fig. 1, the function of the signal input processing unit 213 may be realized by the USB audio I / F 2 shown in Fig. 1 and the CPU 11 of the terminal 1 described later using Fig. 16, and the function of the signal transmitting unit 214 may be realized by the communication I / F (not shown) of the terminal 1 described later using Fig. 16. The function of the signal receiving unit 215 may be realized by the communication I / F (not shown) of the terminal 1 described later using Fig. 16, the function of the signal output processing unit 216 may be realized by the USB audio I / F 2 shown in Fig. 1 and the CPU 11 of the terminal 1 described later using Fig. 16, the function of the signal amplifying unit 217 may be realized by the amplifier 4 shown in Fig. 1, and the function of the vibration presenting unit 218 may be realized by the vibrator 32 shown in Fig. 1.

[0022] The vibration measuring unit 211 measures vibrations in a housing that the user touches. The signal amplifying unit 217 amplifies the signal measured by the vibration measuring unit 211. The signal input processing unit 213 performs signal processing, which will be described later with reference to Figs. 3 to 5 and the like. The signal transmitting unit 214 transmits the signal that has been processed by the signal input processing unit 213 to another location. The haptic signal may be transmitted in synchronization with video and audio. Furthermore, the haptic signal may be transmitted while being included in the audio channel of video or the like.

[0023] The signal receiving unit 215 receives a signal transmitted from another location. The signal output processing unit 216 performs signal processing, which will be described later with reference to Figs. 3 to 5, on the signal received by the signal receiving unit 215. The signal amplifying unit 217 amplifies the signal processed by the signal output processing unit 216. The vibration presentation unit 218 vibrates a housing that the user touches in response to the signal amplified by the signal amplifying unit 217.

[0024] 3A is a block diagram showing a first example of the signal input processing unit 213 shown in FIG. 2, and FIG. 3B is a block diagram showing a first example of the signal output processing unit 216 shown in FIG.

[0025] 3(a), the signal input processing unit 213 has functions as an AD conversion unit 2131, a band-stop filter 2132, an equalizer 2133, and a frequency conversion unit 2134. Moreover, the signal output processing unit 216 has functions as a buffering unit 2161 and a DA conversion unit 2162, as shown in FIG.

[0026] The function of the AD conversion unit 2131 may be realized in the USB audio I / F 2 shown in Fig. 1, and the functions of the band stop filter 2132, the equalizer 2133, and the frequency conversion unit 2134 may be realized in the CPU 11 of the terminal 1, which will be described later with reference to Fig. 16. The function of the buffering 2161 may be realized in the memory 12 or a communication I / F (not shown) of the terminal 1, which will be described later with reference to Fig. 16, and the function of the DA conversion unit 2162 may be realized in the USB audio I / F 2 shown in Fig. 1.

[0027] On the transmitting side, an analog signal is converted into a digital signal by an AD conversion unit 2131, a frequency band used for vibration presentation is removed by a band-stop filter 2132, a signal in a specific band is emphasized by an equalizer 2133, and frequency conversion is performed by a frequency conversion unit 2134, after which the presentation vibration is transmitted to a receiving side. Note that the equalizer 2133 may be omitted.

[0028] On the receiving side, the received presentation vibration is buffered by the buffering 2161, and the digital signal of the presentation vibration is converted into an analog signal by the DA conversion unit 2162.

[0029] FIG. 4(a) is a block diagram showing a second example (see reference symbol 213a) of the signal input processing unit 213 shown in FIG. 2, and FIG. 4(b) is a block diagram showing a second example (see reference symbol 216a) of the signal output processing unit 216 shown in FIG. 2.

[0030] 4(a), the signal input processing unit 213a has functions as an AD conversion unit 2131, a band-stop filter 2132, and an equalizer 2133. Also, the signal output processing unit 216a has functions as a buffering unit 2161, a DA conversion unit 2162, and a frequency conversion unit 2163, as shown in FIG.

[0031] The function of the AD conversion unit 2131 may be realized in the USB audio I / F 2 shown in Fig. 1, and the functions of the band stop filter 2132 and the equalizer 2133 may be realized in the CPU 11 of the terminal 1 described later with reference to Fig. 16. The function of the frequency conversion unit 2163 may be realized in the CPU 11 of the terminal 1 described later with reference to Fig. 16, the function of the buffering 2161 may be realized in the memory 12 of the terminal 1 described later with reference to Fig. 16, and the function of the DA conversion unit 2162 may be realized in the USB audio I / F 2 shown in Fig. 1.

[0032] On the transmitting side, an analog signal is converted into a digital signal by an AD conversion unit 2131, a frequency band used for vibration presentation is removed by a band-stop filter 2132, a signal in a specific band is emphasized by an equalizer 2133, and the signal is transmitted to a receiving side. Note that the equalizer 2133 may be omitted.

[0033] On the receiving side, the frequency conversion unit 2163 performs frequency conversion on the received signal, the buffering unit 2161 buffers the frequency-converted presentation vibration, and the DA conversion unit 2162 converts the digital signal of the presentation vibration into an analog signal.

[0034] FIG. 5(a) is a block diagram showing a third example (see reference symbol 213b) of the signal input processing unit 213 shown in FIG. 2, and FIG. 5(b) is a block diagram showing a third example (see reference symbol 216b) of the signal output processing unit 216 shown in FIG. 2.

[0035] 5(a), the signal input processing unit 213b has functions as an AD conversion unit 2131, a band-stop filter 2132, an equalizer 2133, and an intensity calculation unit 2135. Also, the signal output processing unit 216b has functions as a buffering unit 2161, a DA conversion unit 2162, and a vibration waveform generation unit 2164, as shown in FIG.

[0036] The function of the AD conversion unit 2131 may be realized in the USB audio I / F 2 shown in Fig. 1, and the functions of the band stop filter 2132, the equalizer 2133, and the intensity calculation unit 2135 may be realized in the CPU 11 of the terminal 1 described later with reference to Fig. 16. The function of the vibration waveform generation unit 2164 may be realized in the CPU 11 of the terminal 1 described later with reference to Fig. 16, the function of the buffering 2161 may be realized in the memory 12 of the terminal 1 described later with reference to Fig. 16, and the function of the DA conversion unit 2162 may be realized in the USB audio I / F 2 shown in Fig. 1.

[0037] On the transmitting side, an analog signal is converted to a digital signal by an AD conversion unit 2131, a frequency band used for vibration presentation is removed by a band-stop filter 2132, a specific band of signals is emphasized by an equalizer 2133, and an intensity calculation unit 2135 calculates vibration intensity, and then the signal is transmitted to a receiving side. Since the vibration intensity only requires an update rate of about 100 Hz compared to a waveform signal, it has the effect of reducing the amount of communication. The equalizer 2133 may be omitted.

[0038] On the receiving side, vibration is generated based on the vibration intensity received by the vibration waveform generation unit 2164, the frequency-converted presentation vibration is buffered by the buffering 2161, and the digital signal of the presentation vibration is converted into an analog signal by the DA conversion unit 2162.

[0039] Next, the frequency conversion processing by the frequency conversion units 2134 and 2163 will be described in the following (1) to (3).

[0040] (1) Frequency conversion processing example I 6(a) to 6(c) are graphs for explaining the frequency conversion process by the frequency conversion units 2134 and 2163 shown in FIG. 3 and FIG.

[0041] The frequency conversion units 2134 and 2163 maintain the envelope (in other words, perceptual information) and present it as an amplitude modulated wave. Fig. 6(a) shows the original signal v0(t), Fig. 6(b) shows the upper and lower envelope signals eup(t) and elow(t) of the original signal as dotted lines, and Fig. 6(c) shows the amplitude modulated wave vam(t) generated so as to be included in the envelope of Fig. 6(b). The amplitude modulated wave vam(t) is expressed by the following equation.

number

[0042] where A(t) is the amplitude of the amplitude modulated wave, f is the carrier frequency of the presented amplitude modulated wave, and voff(t) is the offset of the amplitude modulated wave.

[0043] The carrier frequency f is selected to be different from the frequency components of the original signal. It is recommended to select a frequency f of about 150 to 400 Hz, which is easily perceptible to humans. This allows you to feel the fluctuations in the envelope of the original signal.

[0044] (2) Frequency conversion processing example II The frequency conversion units 2134 and 2163 may perform frequency conversion processing by mapping with the equivalent subjective intensity of the stimulus. It is assumed that the subjective intensity of vibration with frequency f and amplitude A is obtained by a function S(A, f). When the representative frequency f0 of the vibration of the original signal can be identified, the amplitude A is obtained so that the subjective intensity S(A0, f0) is equivalent to the subjective intensity S(A, f) of the vibration after the frequency change. The equivalent subjective intensity function may be, for example, an equal sensitivity curve of vibration obtained by the magnitude balance method.

[0045] (3) Frequency conversion processing example III The frequency conversion units 2134 and 2163 may perform frequency conversion processing using only the perceptual intensity expressed by the following equation 2. Using the following equation, the perceptual intensity of the representative frequency f0 of the original signal is calculated, and the amplitude A is calculated so that the perceptual intensity of the vibration after frequency conversion is equivalent.

number

[0046] where A is the amplitude, Tf is the amplitude threshold at frequency f, and bf is an exponent value that depends on frequency f.

[0047] ISM is a method that utilizes time-division perceptual intensity, and may be a method in which time-division processing is added to the processing in the frequency conversion sections 2134 and 2163 described above.

[0048] FIG. 7 is a graph illustrating the measurement results of the amplitude at the through output.

[0049] In the bidirectional tactile transmission system 100 shown in FIG. 1, a verification of feedback was carried out under the following conditions. Sensor: Piezoelectric vibration sensor VS-BV203 Vibrator: Audio Exciter TEAX09C005-8 USB Audio: OCTA-CAPTURE Amplifier: SA-36A PRO Device: 100 mm long, 150 mm wide matte polystyrene board

[0050] In the case of through output, feedback was confirmed as shown in Figure 7. On the other hand, when ISM processing was performed, feedback did not occur even if the sensor sensitivity was increased more than in the case of through output. In other words, it was confirmed that frequency modulation is effective against vibration feedback.

[0051] FIG. 8 is a table illustrating the results of suppressing feedback when ISM is used in the bidirectional tactile transmission system 100 shown in FIG.

[0052] We verified whether feedback would occur by changing the modulation frequency (fm) after conversion with ISM. The natural frequency of the board is about 500Hz, and fm is 200 to 1000Hz.

[0053] In FIG. 8, ◯ indicates that the device does not oscillate even when vibration is applied, △ indicates that the device oscillates when a small amount of vibration is applied, and × indicates that the device self-excites without the application of vibration.

[0054] It was confirmed that when fm was equal to the natural frequency of the board, even slight vibrations would cause feedback, even without touching the board. It was also confirmed that when the frequency was twice the natural frequency, which is a harmonic, applying vibrations to the board would cause feedback. On the other hand, it was confirmed that no feedback would occur when vibrations were applied to the board at other frequencies. In other words, it was confirmed that feedback can be prevented by using ISM to convert vibrations to a frequency away from the natural frequency.

[0055] FIG. 9(a) is a graph illustrating the measurement results of output vibration in the case of through output, and (b) is a graph illustrating the measurement results of output vibration in the case of using ISM.

[0056] ISM can convert vibrations to any carrier frequency, and the vibrations caused by the oscillator are removed by applying a filter to the sensor measurement value that removes the ISM carrier frequency component.

[0057] The frequency after conversion by ISM was set to 200 Hz, and the frequency of 200 Hz was removed by a band-stop filter 2132, and an impulse waveform was presented as the initial vibration.

[0058] As shown in (a) of Fig. 9, in the case of the through output, the initial oscillation shown by symbol G1 was attenuated while loopbacks shown by symbols G2 and G3 were repeated multiple times. On the other hand, as shown in (b) of Fig. 9, in the case of the ISM, no loopback occurred after the initial oscillation shown by symbol G4. This confirmed that the combination of ISM processing and filter processing is effective for loopback.

[0059] Next, specific examples of the vibration input / output devices 3, 3a to 3e will be described with reference to FIGS.

[0060] FIG. 10 is a diagram showing a first example of an environment-installed vibration input / output device 3. As shown in FIG.

[0061] In the vibration input / output device 3 shown in FIG. 10, a vibration sensor 31 and a vibrator 32 (in other words, a vibrator) are disposed on the rear surface of a contact plate 301.

[0062] When the user touches the contact plate 301, the vibration input / output device 3 detects and presents the sense of touch. Two rectangular parallelepiped fixing parts 302 are attached along two sides of the back surface of the contact plate 301, and the contact plate 301 may be fixed to a desk, a notebook computer type terminal 1, a keyboard (not shown), or the like by the fixing parts 302. The contact plate 301 may be fixed at both ends and the middle part may be raised, so that the front surface is easily swayed. An image may be projected so as to be superimposed on the contact plate 301, and an image of the other party may be displayed.

[0063] FIG. 11 is a diagram showing a second example of an environment-installed vibration input / output device 3a.

[0064] 11, like the vibration input / output device 3 shown in FIG. 10, a contact plate 301 is fixed by two fixing parts 302. The vibration input / output device 3a includes one vibration sensor 31 and a plurality of (four in the illustrated example) vibrators 32 on the rear surface of the contact plate 301. In addition, a contact position detection sensor 33 (in other words, a contact position detection sensor) that detects the contact position using a depth camera that tracks the position of the hand with a stereo camera may be separately provided. That is, in the vibration input / output device 3 shown in FIG. 1, the contact position detection sensor 33 may be additionally provided.

[0065] In this way, a sense of vibration movement may be output to the vibrator 32 based on the detection result by the contact position detection sensor 33. For example, vibration is distributed and output to multiple vibrators 32 of the receiving vibration input / output device 3a based on information (e.g., coordinate information) related to the contact position on the contact plate 301 detected by the contact position detection sensor 33. By arranging multiple vibrators 32, the vibration intensity and stimulation time difference of each vibrator 32 can be adjusted according to the contact position of the communication partner, and a sense of movement of the contact object can be virtually expressed.

[0066] FIG. 12 is a diagram showing a third example of an environment-installed vibration input / output device 3b.

[0067] 10 and 11, the vibration input / output device 3c shown in Fig. 12 has a contact plate 301 fixed by two fixing portions 302. The vibration input / output device 3c includes a plurality of vibration sensors 31 (four in the illustrated example) and a plurality of vibrators 32 (four in the illustrated example) on the rear surface of the contact plate 301.

[0068] By providing multiple vibration sensors 31, the contact position can be estimated based on the difference in input strength, and a sense of vibration movement can be output to the vibrator 32 based on the input from each of the multiple vibration sensors 31. Also, by providing multiple vibrators 32, it is possible to vibrate the entire surface of the contact plate. Furthermore, by adjusting the vibration strength and time difference of each vibration according to the contact position of the communication partner, it is possible to virtually express the sense of movement of the contact object.

[0069] FIG. 13 is a diagram showing a wristband-type vibration input / output device 3c.

[0070] The vibration input / output device 3c shown in Fig. 13 includes a plurality of vibration sensors 31 and a plurality of vibrators 32 on the inner surface of a wristband worn by a user on the wrist. The vibration input / output device 3c may be any of various wearable devices that can be worn by a user. Although two vibration sensors 31 and two vibrators 32 are shown in Fig. 13, a plurality of vibration sensors 31 and a plurality of vibrators 32 may be provided so as to surround the user's wrist, for example, four each in the top, bottom, left, and right directions.

[0071] A vibration sensor 31 is attached near the wrist to measure vibrations transmitted from the user's hand when the user touches an object. By using multiple vibration sensors 31, it is possible to obtain differences in vibrations depending on the part of the arm. A vibrator 32 is also attached near the wrist to present vibrations transmitted from the communication partner. By using multiple vibrators 32, it is possible to express differences in vibrations depending on the part of the arm.

[0072] FIG. 14 is a diagram showing a pillow-type vibration input / output device 3d.

[0073] 14, a vibration input / output device 3d has a vibration sensor 31 and a vibrator 32 arranged on a hug pillow, cushion, or the like that can come into contact with the user's body or be held by the user. The number of vibration sensors 31 and vibrators 32 can be changed in various ways. The vibration input / output device 3d comes into contact with a large area of ​​the user's abdomen, legs, or the like, and can measure the vibration of the user's body with the vibration sensor 31 and transmit the vibration to the user's body with the vibrator 32. The vibration sensor 31 and vibrator 32 may be arranged on a handheld controller as a device that can be held by the user.

[0074] FIG. 15 is a diagram showing a tool-mounted vibration input / output device 3e.

[0075] 15, a vibration input / output device 3e has a vibration sensor 31 and a vibrator 32 arranged on a tool that can be held by a user, such as a wrench or a screwdriver. The number of vibration sensors 31 and vibrators 32 can be changed in various ways. The vibration sensor 31 measures the vibration transmitted to the tool, and the vibrator 32 is attached to the tool to stimulate the hand, and the stimulation is transmitted in both directions to convey the feeling of operation to a user in a remote location, making it possible to provide remote technical guidance, etc.

[0076] In the vibration input / output devices 3, 3a to 3e shown in FIGS. 10 to 15, at least one of sound, video, and light may be output simultaneously with output of the converted signal by the vibrator 32.

[0077] [A-2] Terminal FIG. 16 is a block diagram illustrating a schematic configuration example of a terminal 1 according to an embodiment.

[0078] The terminal 1 includes a Central Processing Unit (CPU) 11 , a memory 12 , and a storage device 13 .

[0079] The memory 12 is a storage device including a Read Only Memory (ROM) and a Random Access Memory (RAM).

[0080] The storage device 13 is a device that stores data in a readable and writable manner, and may be, for example, a hard disk drive (HDD), a solid state drive (SSD), or a storage class memory (SCM). The storage device 13 stores generated teacher data, learning models, and the like.

[0081] The CPU 11 is a processing device that performs various controls and calculations, and realizes various functions by executing an Operating System (OS) and programs stored in the memory 12. That is, the CPU 11 may function as a calculation unit 113A, a conversion unit 114A, and a signal output unit 115A, as shown in FIG.

[0082] The CPU 11 is an example of a computer, and exemplarily controls the operation of the entire terminal 1. The device for controlling the operation of the entire terminal 1 is not limited to the CPU 11, and may be, for example, any one of an MPU, a DSP, an ASIC, a PLD, an FPGA, and a dedicated processor. The device for controlling the operation of the entire terminal 1 may be a combination of two or more of the CPU, the MPU, the DSP, the ASIC, the PLD, the FPGA, and a dedicated processor. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. Also, PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

[0083] The calculation unit 113A calculates perceptual information identified from the vibration measured by the vibration sensor 31. The calculation unit 113A may calculate an envelope identified from the vibration measured by the vibration sensor 31 as the perceptual information. The calculation unit 113A may calculate a subjective intensity of a stimulus as the perceptual information from a waveform of the vibration measured by the vibration sensor 31. The calculation unit 113A may calculate a perceptual intensity (in other words, the energy of a signal) identified from the vibration measured by the vibration sensor 31 as the perceptual information. The calculation unit 113A may divide a signal related to the vibration measured by the vibration sensor 31 for each predetermined time period, and calculate a perceptual intensity for each divided predetermined time period.

[0084] The conversion unit 114 converts the signal related to vibration to a predetermined frequency while maintaining the perceptual information calculated by the calculation unit 113A. The conversion unit 114A may convert the signal to another frequency other than the resonant frequency of the housing of the vibration input / output device 3. The conversion unit 114A may output the signal related to vibration as an amplitude modulated signal converted so that a predetermined frequency becomes a carrier frequency while maintaining the envelope calculated by the calculation unit 113A. The conversion unit 114A may convert the signal related to vibration to a predetermined frequency using a predetermined equivalent subjective intensity map so as to maintain the subjective intensity calculated by the calculation unit 113A. The conversion unit 114A may convert the signal related to vibration to a waveform having a different frequency while maintaining the perceptual intensity calculated by the calculation unit 113A. The conversion unit 114A may convert the signal related to vibration to a waveform having a different frequency by adjusting the perceptual intensity for a signal in a specific frequency band among the frequency components of the signal, while converting a signal other than the specific frequency band to a waveform having a different frequency while maintaining the perceptual intensity. The conversion unit 114A may convert the waveform of a signal extracted based on a specific signal feature by adjusting the perceived intensity, while converting a signal not extracted by the specific feature into a waveform having a different frequency while maintaining the perceived intensity. The conversion unit 114A may attenuate a specific frequency band by filtering so as to suppress loopback of the vibration output from the vibrator 32. The filtering may be realized by the band-stop filter 2132 shown in Figs. 3 to 5.

[0085] The signal output unit 115A outputs the signal converted by the conversion unit 114A as output vibration to the vibrator 32 of the other vibration input / output device 3. The signal output unit 115A may output the amplitude modulated signal converted by the conversion unit 114A to the vibrator 32 of the other vibration input / output device 3.

[0086] Of the functions of the calculation unit 113A, the conversion unit 114A, and the signal output unit 115A, any one of a combination of only the calculation unit 113A, a combination of the calculation unit 113A and the conversion unit 114A, and a combination of the calculation unit 113A, the conversion unit 114A, and the signal output unit 115A may be provided in the transmitting terminal 1. Also, of the functions of the calculation unit 113A, the conversion unit 114A, and the signal output unit 115A, any one of a combination of only the signal output unit 115A, a combination of the conversion unit 114A and the signal output unit 115A, and a combination of the calculation unit 113A, the conversion unit 114A, and the signal output unit 115A may be provided in the receiving terminal 1.

[0087] [A-3] ISM 17(a) to 17(c) are graphs briefly explaining the ISM processing used in the distribution processing of the perceptual intensity shown in FIG.

[0088] ISM is a technique for modulating high-frequency vibration to low frequency while maintaining the tactile sensation. The original signal shown in Fig. 17(a) is transformed to calculate the vibration intensity for each segment shown in Fig. 17(b). Then, the transformed waveform shown in Fig. 17(c) is generated while maintaining the vibration intensity.

[0089] In FIG. 17, the vibration is 400 to 600 Hz before conversion, whereas the waveform after conversion is 200 Hz, but any frequency can be selected as the waveform after conversion.

[0090] In generating the high frequency component signal, a vibration waveform equivalent to the distributed perceived intensity is generated. Simply put, since the waveforms of each vibrator 32 have the same frequency, the original waveform may be multiplied by a gain value calculated from the distribution coefficient to drive the vibrator (the same method as for the low frequency components described later). However, the tactile vibrator 32 generally has a narrow response frequency band, and it is difficult to generate an arbitrary vibration waveform as is. In addition, when an acoustic signal is used as the waveform of the vibration source, since it contains frequencies in the audible range, there is a problem that noise is generated when the vibrator 32 is driven.

[0091] Therefore, the signal is converted into an amplitude modulated wave with an appropriate carrier frequency so as to generate a distributed perceived intensity Ik. This results in a single carrier frequency for the generated signal. The carrier frequency can be selected according to the frequency response characteristics of the transducer. Considering the human perception characteristics of high-frequency vibration, a carrier frequency in the range of 150 to 400 Hz is appropriate.

[0092] Taking into account the human perception characteristics of high-frequency vibrations, in the high-frequency band, rather than focusing on the waveform itself, we focus on the vibration energy that is correlated with human perception characteristics, and by replacing it with another waveform that has the same vibration energy, the frequency band can be changed.

[0093] By dividing any continuous vibration signal into appropriate time intervals that take into account human perception characteristics and converting each divided segment into vibration energy, it is possible to convert the signal into any waveform while maintaining the same tactile sensation felt by humans, or to enable the sensation of high frequencies that are difficult to sense.

[0094] By appropriately selecting the frequency of the converted vibration, it is possible to efficiently drive the transducer in accordance with its response range, reduce auditory noise, or convert the transducer into any sound source.

[0095] It is said that humans can only perceive vibrations up to about 1 kHz. Therefore, vibrations above 1 kHz are often ignored. However, it is known that even if the vibration is above 1 kHz, if it is an amplitude modulated wave whose amplitude fluctuates within a band that can be sensed by humans, the envelope component can be perceived.

[0096] On the other hand, a vibration energy model is known as a human perception characteristic for high-frequency vibrations of about 100 Hz or higher. From this, it is known that vibrations cannot be distinguished even if the carrier frequency of the amplitude-modulated wave is replaced while maintaining the high-frequency vibration energy. However, even if the vibration energy is maintained, as mentioned above, there are cases where the envelope component of the vibration can be perceived as a difference in tactile information, and the range of perception has not been investigated. In addition, although a method has been devised to convert signals based on vibration energy by time division, a method for maintaining low-frequency components has not been considered.

[0097] Fig. 18 is a graph showing the discriminability of vibrations by humans. Fig. 19 is a sample waveform of vibrations used in a forced three-choice discrimination experiment conducted to determine the discriminability shown in the graph in Fig. 18.

[0098] Based on the assumption of a conventionally known vibration energy model, the graph shown in FIG. 18 is obtained by investigating the human perceptual discrimination characteristics while maintaining the vibration energy. Symbols B1 and B2 in FIG. 19 show the same waveform, and symbol B3 in FIG. 19 shows a different waveform. The subjects are asked to compare the constant amplitude vibration shown in symbols B1 and B2 in FIG. 19 with the amplitude modulated stimulus shown in symbol B3, and to answer which is the amplitude modulated wave. In FIG. 18, the correct answer rate obtained in the forced three-choice discrimination experiment is expressed as Sensitivity (d': d-prime), which is a discrimination performance index based on signal detection theory, and it means that when d' is 1 or less, the correct answer rate falls below about 60%.

[0099] According to the graph in Fig. 18, the upper limit of the frequency at which the envelope components can be discriminated is about 80 to 125 Hz. It is also shown that there is no need to preserve the envelope components above this upper limit of frequency, and that if the carrier frequency of the amplitude modulated wave is replaced while maintaining the vibration energy, the stimuli cannot be discriminated.

[0100] As mentioned above, even if the vibration energy is maintained, if the energy fluctuates in the low-frequency range, the fluctuations may be perceived as differences in tactile information, but the range of perception has not been investigated. Based on the discovery that the upper limit of perceptible low-frequency fluctuations is approximately 80 to 125 Hz, we decided to convert the vibration energy while maintaining the low-frequency components through two measures (see measures [1] and [2] below).

[0101] FIG. 20 is a graph showing signal waveforms before and after conversion for each segment by terminal 1 shown in FIG.

[0102] Human high frequency perception is based on vibration energy rather than the waveform itself, so if the vibration energy is maintained, the same sensation will be felt. However, if the vibration energy fluctuation occurs below about 80-125Hz, it is necessary to reproduce the vibration energy fluctuation.

[0103] Therefore, in one example of this embodiment, as a means for maintaining the fluctuation of vibration energy below a predetermined frequency (e.g., about 80 to 125 Hz), the vibration is divided into time intervals, for example, about 80 to 200 Hz, the vibration energy is obtained for each segment, and it is replaced with a vibration having a different carrier frequency.

[0104] In the example shown in Figure 20, the original vibration signal shown with reference symbol C1 and the transformed signal shown with reference symbol C2 are transformed so that the energy of the transformed signal is the same as the energy of the original vibration signal within the same time segment.

[0105] The width of the time division (in other words, the division width) should be set to a level that can express energy fluctuations of 80 to 125 Hz or less (in other words, to a level where the peaks of the fluctuations are aligned) (Measure [1]). The frequency of the division width may be 80 to 125 Hz or more, but if the division width is made too short, the estimation accuracy of vibration energy with a period longer than the division width will deteriorate. Therefore, by taking the following measure [2], the waveform of vibrations whose energy cannot be estimated is output as is.

[0106] Also, components below a predetermined frequency may be extracted and presented as they are as vibration stimulation (measure [2]). The predetermined frequency may be 80 to 125 Hz or more, but components above the predetermined frequency may be expressed by the energy control unit 113 of the second signal component. This allows for arbitrary frequency selection. However, if the predetermined frequency is set too high, noise problems may occur or a wideband vibration device may be required.

[0107] According to the above measures [1] and [2], the predetermined frequency may be about 80 to 400 Hz. 400 Hz is the upper limit from the viewpoint of noise issues and the performance of the vibration device.

[0108] The setting of the predetermined frequency also involves the selection of the carrier frequency when converting the vibration. Since the peak of the vibration frequency at which human perception improves is around 200 to 250 Hz, a practical carrier frequency that increases sensitivity without generating noise is about 150 to 400 Hz. The carrier frequency may be a constant multiple of the division width. In addition, the carrier frequency may use a plurality of different frequencies, and may include a high frequency range of 400 Hz or higher.

[0109] Furthermore, the predetermined frequency that separates low and high frequencies does not necessarily have to match the frequency of the division width for calculating energy.

[0110] The corrected energy, which is the vibration energy corrected to enhance human perceptibility, can be expressed as follows:

[0111]

number

[0112] FIG. 21 is a graph showing the amplitude threshold Tf used in calculating the correction energy.

[0113] As shown in FIG. 21, the amplitude threshold varies depending on the frequency. In the range of approximately 102 to 103 Hz, humans can sense even relatively small amplitudes, but in other ranges, humans can only sense relatively large amplitudes.

[0114] FIG. 22 is a graph showing the exponent value bf used in calculating the correction energy.

[0115] The exponent value bf in FIG. 22 is an example in which a value obtained by linearly interpolating the exponent value bf of 400 Hz or less that has been previously reported is used.

[0116] FIG. 23 is a diagram for explaining the use of a window function in the terminal 1 shown in FIG.

[0117] As shown by D1, a high-frequency signal H(t) is input. As shown by D2, the high-frequency signal H(t) is divided into frames i, i+1, i+2,... as signals hi, hi+1, hi+2,... As shown by D3, the signal h of each divided frame is separated into multiple basis signals g1, g2, g3... As shown by D4, scalar values ​​Ei, Ei+1, Ei+2,... that combine the correction energies of all the basis signals g1, g2, g3... are output based on the frequencies f1, f2, f3... of the basis signals g1, g2, g3... As shown by reference symbol D5, the scalar values ​​Ei, Ei+1, Ei+2,... of the vibration energy calculated for each frame i are converted into a vibration waveform having the same vibration energy but a different carrier frequency, and a windowing process is performed using a window function on the amplitudes ai(t), ai+1(t), ai+2(t),... of the waveform. As shown by reference symbol D6, frame synthesis is performed for the 1st to Nth frames, and the amplitude A(t) of the vibration waveform is output. As shown by reference symbol D7, a second vibration waveform S2(t) having a carrier frequency such that the amplitude is A(t) is output.

[0118] FIG. 24 is a graph illustrating an example of combining a low frequency and a high frequency in terminal 1 shown in FIG.

[0119] The second vibration waveform S2(t) shown in E1, which is generated from the high-frequency signal H(t) using the window function in Fig. 23, is synthesized with the first vibration waveform S1(t) shown in E2, which is the low-frequency signal L(t) output as is. As a result, the synthesized waveform S1(t)+S2(t) shown in E3 is output.

[0120] FIG. 25 is a graph showing a specific example of signal waveforms before and after conversion by terminal 1 shown in FIG.

[0121] In FIG. 25, the waveform of the violin sound before conversion (see symbol F1) and the waveform after conversion (see symbol F2) are represented by amplitude over time.

[0122] For high-frequency vibrations such as those of a violin, conventional tactile vibration generates a lot of auditory noise, and applying a low-pass filter eliminates the vibrations that humans can perceive. Therefore, the correction energy is calculated so that the waveform becomes a single wavelength with a low-frequency carrier frequency per time.

[0123] FIG. 26 is a block diagram illustrating an example of a functional configuration of the ISM unit 1000 in the terminal 1 shown in FIG.

[0124] The ISM unit 1000 functions as a time division control unit 112, an energy control unit 113, an energy vibration conversion unit 114a, and a vibration generation unit 114b. In this embodiment, the ISM unit 1000 controls the vibration including high frequency components of about 100 Hz or more by the vibrator 32 using a signal. The methods of controlling vibration including high frequency components of 100 Hz or more according to the present invention are collectively called ISM.

[0125] The time division control unit 112 divides the vibration signal X(t) containing high frequency components of about 100 Hz or more into N frames in time, and inputs the time-divided i-th frame signal hi to the energy control unit 113. The number of frames N may be determined by a predetermined period and an overlap rate of the windowing process.

[0126] The energy control unit 113 calculates a corrected energy ei for the signal hi of the i-th frame, and inputs the calculated corrected energy to the energy vibration conversion unit 114a.

[0127] The energy vibration conversion section 114a generates a signal A(t) by combining the corrected energies e1 to eN of the 1st to Nth frames, and inputs the signal A(t) to the second vibration generation section 114b.

[0128] The vibration generating unit 114b outputs a signal waveform S(t) based on the synthesized signal A(t).

[0129] A first embodiment of the vibration waveform generation process in the terminal 1 shown in FIG. 16 will be described with reference to the block diagram (steps S1 to S7) shown in FIG.

[0130] The signal removal unit 111a removes components below a predetermined frequency from the acquired pre-conversion signal X(t) to generate a high-frequency signal H(t), and inputs the high-frequency signal H(t) to the time division control unit 112 (step S1).

[0131] The time division control unit 112 divides the high frequency signal H(t) into N frames and inputs the time-divided i-th frame signal hi to the energy control unit 113 (step S2). The number of frames N may be determined by a predetermined period and an overlap rate of the windowing process.

[0132] The energy control unit 113 calculates a corrected energy ei for the signal hi of the i-th frame, and inputs the calculated corrected energy to the energy vibration conversion unit 114a (step S3).

[0133] The energy vibration conversion unit 114a generates a signal A(t) by combining the corrected energies e1 to eN of the 1st to Nth frames, and inputs the signal A(t) to the second vibration generation unit 114b (step S4).

[0134] The second vibration generating unit 114b outputs a second vibration waveform S2(t) based on the synthesized signal A(t) (step S5).

[0135] On the other hand, the low-pass filter 111b inputs a low-pass signal L(t) obtained by filtering out components having a frequency equal to or lower than a predetermined frequency from the acquired pre-conversion signal X(t) to the first vibration generating unit 114c (step S6).

[0136] The first vibration generating unit 114c outputs a first vibration waveform S1(t) based on the low-frequency signal L(t) (step S7).

[0137] Next, the energy control process shown in step S3 of FIG. 27 will be described in detail with reference to the block diagram shown in FIG. 28 (steps S11 to S14).

[0138] As shown in FIG. 28, the energy control unit 113 functions as a basis signal separation control unit 113a, a frequency calculation unit 113b, an energy correction parameter calculation unit 113c, and a corrected energy calculation unit 113d.

[0139] The basis signal separation control unit 113a separates the time-divided i-th frame signal hi, which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal gk to the frequency calculation unit 113b (step S11). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, Empirical Mode Decomposition (EMD) method, or the like.

[0140] The frequency calculation unit 113b calculates the frequency fk of the k-th basis signal gk by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs it to the energy correction parameter calculation unit 113c (step S12).

[0141] The energy correction parameter calculation unit 113c calculates the exponent value bk and the amplitude threshold value Tk described with reference to Figs. 21 and 22 based on the frequency fk, and inputs them to the corrected energy calculation unit 113d (step S13).

[0142] The corrected energy calculation unit 113d calculates the corrected energy Ipc for each basis signal gk based on the exponent value bk and the amplitude threshold Tk according to the formula shown in Equation 3, and outputs a scalar value ei that is the sum of the corrected energies of all the basis signals gk (step S14).

[0143] Next, as a second example of the vibration waveform generation process in terminal 1 shown in FIG. 16, the separation process of low-frequency components in the energy control process shown in FIG. 26 will be described with reference to the block diagram (steps S101 to S105) shown in FIG. 29.

[0144] As shown in FIG. 29, the energy control unit 113 functions as a basis signal separation control unit 113a, a frequency calculation unit 113b, an energy correction parameter calculation unit 113c, and a corrected energy calculation unit 113d, and may also have a function of separating low-frequency components to a low-frequency component synthesis unit 113g.

[0145] The basis signal separation control unit 113a separates the time-divided i-th frame signal hi, which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal gk to the frequency calculation unit 113b (step S101). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

[0146] The frequency calculation unit 113b calculates the frequency fk of the k-th basis signal gk by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs it to the energy correction parameter calculation unit 113c (step S102).

[0147] The energy correction parameter calculation unit 113c calculates the exponent value bk and the amplitude threshold value Tk described with reference to Figs. 21 and 22 based on the frequency fk, and inputs them to the corrected energy calculation unit 113d (step S103).

[0148] The corrected energy calculation unit 113d calculates the corrected energy Ipc for each basis signal gk based on the exponent value bk and the amplitude threshold Tk according to the formula shown in Equation 3, and outputs a scalar value ei that is the sum of the corrected energies of all the basis signals gk (step S104).

[0149] The low-frequency component synthesis unit 113g synthesizes a base signal in which the frequency fk of the base signal gk is lower than a predetermined frequency, and generates a low-frequency component L(t) (step S105).

[0150] There are cases where it is desired to emphasize vibration energy in a specific frequency band and present it as vibration for a sound source including signals in multiple frequency bands. In such a case, energy control units 1131 and 1132 are used as modified examples to adjust the energy of a base signal present in a predetermined frequency band and convert the waveform, as will be described with reference to Figs. 30 to 35.

[0151] (a) to (c) of Fig. 30 are graphs explaining an example of generating vibrations according to ISM without emphasizing the waveform. Fig. 30 shows a band corresponding to the waveform of the cymbal (drum) of the high frequency components from a piano trio song, and bands corresponding to the waveforms of the piano and bass. In (a) to (c) of Fig. 30, the horizontal axis indicates time [s], and the vertical axis indicates frequency [Hz], with darker spectra indicating greater power and lighter spectra indicating less power.

[0152] FIG. 30(a) shows, as a distribution of the sound source spectrum, the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed-dotted line.

[0153] Figure 30(b) shows the spectral distribution (centered at 200 Hz) after conversion with ISM. In Figure 30(b), the cymbals, piano, and bass are all extracted as intensities due to the effect of ISM.

[0154] Figure 30(c) shows an example of converting the signal to a signal with a representative frequency of the base signal, rather than converting it to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands are emphasized.

[0155] Fig. 31 (a) to (c) are graphs for explaining a first example of emphasizing and separating high frequency components from a sound source. Fig. 31 shows an example of emphasizing and separating a high frequency component of a cymbal (drum) from a piano trio piece of music. In Fig. 31 (a) to (c), the horizontal axis indicates time [s] and the vertical axis indicates frequency [Hz], with a darker spectrum indicating greater power and a lighter spectrum indicating less power.

[0156] FIG. 31(a) shows, as a distribution of the sound source spectrum, the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed-dotted line.

[0157] Figure 31(b) shows the spectrum distribution (centered at 200 Hz) after conversion using ISM. In Figure 31(b), only the intensity above 3000 Hz is increased by +20 dB (100 times).

[0158] Figure 31(c) shows an example of converting the signal to a signal with a representative frequency of the base signal, rather than converting it to a signal with a frequency of 200 Hz based on the intensity. This visualizes which frequency bands are emphasized. In Figure 31(c), the power of the cymbal spectrum has increased.

[0159] (a) to (c) of Fig. 32 are graphs for explaining a second example of emphasizing and separating high frequency components from a sound source. Fig. 32 shows an example of emphasizing and separating a high frequency component of a cymbal (drum) from a piano trio piece of music. In (a) to (c) of Fig. 32, the horizontal axis indicates time [s], the vertical axis indicates frequency [Hz], and a darker spectrum indicates a larger power, and a lighter spectrum indicates a smaller power.

[0160] FIG. 32(a) shows, as a distribution of the sound source spectrum, the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed-dotted line.

[0161] Figure 32(b) shows the spectrum distribution (centered at 200 Hz) after conversion using ISM. In Figure 32(b), the intensity above 3000 Hz is increased by +20 dB (100 times), while the intensity below 1000 Hz is decreased by -10 dB (1 / 10 times).

[0162] Figure 32(c) shows an example of converting the signal to a signal with a representative frequency of the base signal, rather than converting it to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands are emphasized. In Figure 32(c), the power of the cymbal spectrum has increased.

[0163] (a) to (c) of Fig. 33 are graphs for explaining an example of emphasizing and separating low-frequency components from a sound source. Fig. 33 shows an example of emphasizing and separating low-frequency components of piano and bass from a piano trio piece of music. In (a) to (c) of Fig. 33, the horizontal axis indicates time [s], the vertical axis indicates frequency [Hz], and a darker spectrum indicates a larger power, and a lighter spectrum indicates a smaller power.

[0164] FIG. 33(a) shows, as a distribution of the sound source spectrum, the waveform of the cymbal, which is a high-frequency component, indicated by a dashed line, and the waveforms of the piano and bass, which are low-frequency components, indicated by a dashed-dotted line.

[0165] Figure 33(b) shows the spectrum distribution (centered at 200 Hz) after conversion using ISM. In Figure 33(b), the intensity below 1000 Hz has been increased by +10 dB (10 times).

[0166] Figure 33(c) shows an example of converting the signal to a signal with a representative frequency of the base signal, rather than converting it to a signal with a frequency of 200 Hz based on the intensity. This makes it possible to visualize which frequency bands are emphasized. In Figure 33(c), the power of the piano and bass spectra is increased.

[0167] The process of emphasizing and separating an arbitrary frequency component shown in Fig. 30 to Fig. 33 can also be applied to the bidirectional tactile transmission system 100 shown in Fig. 1. That is, a signal in a specific frequency band among the frequency components of a signal may be converted into a waveform by adjusting the energy, while a signal outside the specific frequency band may be converted into a waveform having a different frequency while maintaining the energy. Also, a signal extracted based on a specific signal feature may be converted into a waveform having a different frequency while maintaining the energy, while a signal not extracted by the specific feature may be converted into a waveform having a different frequency while maintaining the energy.

[0168] A first modified example of the energy control process shown in FIG. 26 will be described with reference to the block diagram shown in FIG. 34 (steps S41 to S45).

[0169] As shown in FIG. 34, an energy control unit 1131 functions as a gain calculation unit 113e in addition to the basis signal separation control unit 113a, the frequency calculation unit 113b, the energy correction parameter calculation unit 113c, and the corrected energy calculation unit 113d shown in FIG.

[0170] The basis signal separation control unit 113a separates the time-divided i-th frame signal hi, which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal gk to the frequency calculation unit 113b (step S41). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

[0171] The frequency calculation unit 113b calculates the frequency fk of the k-th basis signal gk by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs it to the energy correction parameter calculation unit 113c (step S42).

[0172] The energy correction parameter calculation unit 113c calculates the exponent value bk and the amplitude threshold value Tk described with reference to Figs. 21 and 22 based on the frequency fk, and inputs them to the corrected energy calculation unit 113d (step S43).

[0173] The gain calculation unit 113e outputs a gain value Gk for each predetermined frequency band according to the frequency fk of the calculated base signal gk (step S44). When it is desired to emphasize the energy, Gk>1 is set, and when it is desired to suppress the energy, 0≦Gk<1 is set. The energy adjustment by emphasis or suppression may be performed for one frequency band or for multiple frequency bands. Moreover, the energy adjustment may be performed for the entire frequency band input to the energy control unit 1131.

[0174] The corrected energy calculation unit 113d calculates a gain-adjusted corrected energy Ipc for each separated base signal gk with respect to the amplitude A of the base signal gk according to the following formula 2, and outputs a scalar value ei that is the sum of the corrected energies of all the base signals gk (step S45).

[0175]

number

[0176] As shown in FIG. 35, the energy control unit 1132 functions as a gain calculation unit 113e and a signal source identification unit 113f in addition to the basis signal separation control unit 113a, the frequency calculation unit 113b, the energy correction parameter calculation unit 113c, and the corrected energy calculation unit 113d shown in FIG. 28.

[0177] The basis signal separation control unit 113a separates the time-divided i-th frame signal hi, which is an input signal, into a plurality of basis signals g, and inputs the separated k-th basis signal gk to the frequency calculation unit 113b (step S51). For example, the signals may be separated by short-time Fourier analysis, wavelet analysis, EMD method, or the like.

[0178] The frequency calculation unit 113b calculates the frequency fk of the k-th basis signal gk by, for example, discrete Fourier analysis or Hilbert Spectrum analysis, and inputs it to the energy correction parameter calculation unit 113c (step S52).

[0179] The energy correction parameter calculation unit 113c calculates the exponent value bk and the amplitude threshold value Tk described with reference to Figs. 21 and 22 based on the frequency fk, and inputs them to the corrected energy calculation unit 113d (step S53).

[0180] The signal source identification unit 113f estimates identification candidates from the input signals hi and hi history based on the set signal features, identifies which signal source the basis signal gk belongs to, and outputs the identification result as an ID (identifier) ​​or the like (step S54). The signal source identification unit 113f may prepare a classifier in advance using machine learning or the like. For example, the features of many musical instruments may be learned by deep learning, and a group of candidates (e.g., piano, bass, drums) may be estimated as to which musical instrument is included in the current input signal hi (or the history of each of the multiple input signals hi when the input signal hi is too short), and it may be identified as to which musical instrument the basis signal gk belongs to.

[0181] The gain calculation unit 113e outputs a gain value Gk for each predetermined frequency band according to the ID identified by the signal source identification unit 113f (step S55). When it is desired to emphasize the energy, Gk is set to Gk>1, and when it is desired to suppress the energy, Gk is set to 0≦Gk<1. The energy adjustment by emphasis or suppression may be performed for one frequency band or for multiple frequency bands. Moreover, the energy adjustment may be performed for the entire frequency band input to the energy control unit 1132.

[0182] The corrected energy calculation unit 113d calculates a gain-adjusted corrected energy Ipc for each separated base signal gk with respect to the amplitude A of the base signal gk according to the formula shown in Equation 4, and outputs a scalar value ei which is the sum of the corrected energies of all the base signals gk (step S56).

[0183] Next, the energy synthesis process shown in step S4 of FIG. 26 will be described in detail with reference to the block diagram shown in FIG. 36 (steps S21 to S23).

[0184] The energy vibration converter 114a functions as an energy equivalent converter 1141a, a windowing processor 1142a, and a frame synthesizer 1143a.

[0185] As shown in FIG. 36, the energy equivalent conversion unit 1141a converts the scalar value ei of the vibration energy calculated for each frame i into a vibration waveform having an equivalent vibration energy but a different carrier frequency, and outputs the amplitude ai(t) of the waveform to the windowing processing unit 1142a (step S21).

[0186] The windowing unit 1142a performs windowing processing on the input amplitude ai(t) of each frame i using the window function shown in FIG. 23, and inputs the processing result to the frame synthesis unit 1143a (step S22).

[0187] The frame synthesis unit 1143a performs frame synthesis on the input from the windowing processing unit 1142a for the 1st to Nth frames, and outputs the amplitude A(t) of the vibration waveform (step S23).

[0188] Next, the details of the process of generating the corrected vibration waveform shown in step S5 of FIG. 26 will be described with reference to the block diagram shown in FIG. 37 (steps S31 and S32).

[0189] 37, the second vibration generating unit 114b functions as an amplitude vibration converting unit 1141b and a waveform output unit 1142b. The second vibration generating unit 114b has an input signal A(t) and outputs a sine wave having a carrier frequency. The generated waveform may be phase-controlled so that the vibrations are smoothly connected.

[0190] The amplitude vibration converter 1141b converts the input amplitude A(t) into vibration (step S31).

[0191] The waveform output section 1142b outputs a sine wave S2(t) having a carrier frequency such that the amplitude becomes A(t) (step S32).

[0192] [B] Effect According to the bidirectional tactile transmission system 100, the tactile transmission program, and the tactile transmission method in one example of the embodiment, for example, the following advantageous effects can be achieved.

[0193] The vibration sensor 31 measures vibrations generated in the vibration input / output device 3. The calculation unit 113A calculates sensory information identified from the vibrations measured by the vibration sensor 31. The conversion unit 114A converts a signal corresponding to the vibration into a predetermined frequency while maintaining the sensory information calculated by the calculation unit 113A. The signal output unit 115A causes the vibrator 32 of the other vibration input / output device 3 to output the signal converted by the conversion unit 114A as an output vibration.

[0194] This makes it possible to suppress howling and loopback of the contact signal in the bidirectional tactile transmission system 100. Specifically, by modulating the actuator signal to a signal having a different frequency from the natural vibration of the housing while maintaining the sensory information, it becomes possible to suppress howling caused by bidirectional communication. In addition, by keeping the actuator drive signal constant at a carrier frequency different from the natural vibration of the housing and removing the signal having this carrier frequency from the sensor signal, it becomes possible to prevent interference with the sensor signal and suppress loopback.

[0195] [C] Other The disclosed technology is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the embodiments. The configurations and processes of the embodiments can be selected as necessary, or can be combined as appropriate.

[0196] FIG. 38 is a block diagram showing an example of the configuration of a DAC when a plurality of vibration devices 310 and 320 are used in the bidirectional tactile transmission system 100 shown in FIG.

[0197] 38, the DA conversion unit 2162 shown in Fig. 3 to Fig. 5 functions as the high-frequency gain adjuster 21a, the low-frequency gain adjuster 21b, the high-frequency vibration device drive circuit 22a, and the low-frequency vibration device drive circuit 22b. Also, the vibrator 32 shown in Fig. 1 functions as the high-frequency vibration device 310 and the low-frequency vibration device 320.

[0198] The high-frequency gain adjuster 21a outputs the second vibration waveform S2(t) input from the terminal 1 to the high-frequency vibration device 310 via the high-frequency vibration device drive circuit 22a. The low-frequency gain adjuster 21b outputs the first vibration waveform S1(t) input from the terminal 1 to the low-frequency vibration device 320 via the low-frequency vibration device drive circuit 22b.

[0199] FIG. 39 is a block diagram showing an example of the configuration of a DAC in the bidirectional tactile transmission system 100 shown in FIG. 1 when a single vibration device is used.

[0200] 39, the DA conversion unit 2162 shown in FIGS. 3 to 5 functions as the high-frequency gain adjuster 21a, the low-frequency gain adjuster 21b, and the vibration device driving circuit 22. In addition, the vibrator 32 shown in FIG.

[0201] The high-frequency gain adjuster 21a and the low-frequency gain adjuster 21b output the second vibration waveform S2(t) and the first vibration waveform S1(t) input from the terminal 1 to a common vibration device 30 via a common vibration device driving circuit 22, respectively. [Explanation of symbols]

[0202] 100: Two-way tactile transmission system 1: Terminal 11: CPU 1000:ISM Department 111: Frequency elimination control section 111a: Signal removal section 111b:Low pass filter 111d: Correction energy calculation unit 112: Time division control section 113A: Calculation section 113, 1131, 1132: Energy control unit 113a: Basis signal separation control section 113b: Frequency calculation unit 113c: Energy correction parameter calculation unit 113d: Correction energy calculation unit 113e: Gain calculation unit 113f: Signal source identification section 113g: Low frequency component synthesis section 114A: Converter 114a: Energy vibration conversion section 114b: Second vibration generation section 114c: 1st vibration generation section 1141a: Energy equivalent conversion section 1142a: Window processing unit 1143a: Frame synthesis section 1141b: Amplitude vibration converter 1142b: Waveform output section 115A: Signal output section 12: Memory 13:Storage device 2: USB audio interface 21a: High frequency gain adjuster 21b: Low-frequency gain adjuster 211: Vibration measurement section 212, 217: Signal amplifier 213: Signal input processing unit 2131: AD conversion section 2132: Band-stop filter 2133: Equalizer 2134, 2163: Frequency conversion section 2135: Intensity calculation section 214: Signal transmission unit 215: Signal receiving unit 216: Signal output section 2161: Buffering 2162: DA conversion section 2164: Vibration waveform generation section 22: Vibration device drive circuit 22a: High-frequency vibration device driver circuit 22b: Low-frequency vibration device driver circuit 3, 3a to 3e: Vibration input / output device 301: Contact plate 302:Fixed part 31: Vibration sensor 310: High-frequency vibration device 32: Vibrator 320: Low-frequency vibration device 33: Contact position detection sensor 4: Amplifier

Claims

1. A bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, a vibration measuring unit that measures vibrations generated in the tactile transmission device; a calculation unit that calculates perceptual information specified from the vibration measured by the vibration measuring unit; a conversion unit that converts the signal related to the vibration into a predetermined frequency while maintaining the sensory information calculated by the calculation unit; a signal output unit that outputs the converted signal from the conversion unit to a vibrator of the other tactile transmission device as an output vibration; Equipped with the conversion unit attenuates a specific frequency band by filtering so as to suppress loopback of the vibration output from the vibrator. Two-way tactile transmission system.

2. the conversion unit converts the signal into a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. The bidirectional tactile transmission system according to claim 1 .

3. The calculation unit calculates, as the perception information, an envelope curve specified from the vibration measured by the vibration measurement unit; the conversion unit outputs the signal related to the vibration as an amplitude modulated signal converted so that the predetermined frequency becomes a carrier frequency while maintaining the envelope calculated by the calculation unit; the signal output unit outputs the amplitude modulated signal converted by the conversion unit to a vibrator of the other tactile transmission device. The bidirectional tactile transmission system according to claim 1 or 2.

4. The calculation unit calculates a subjective intensity of a stimulus as the perceptual information from a waveform of the vibration measured by the vibration measurement unit; The conversion unit converts the signal related to the vibration into the predetermined frequency by utilizing a predetermined equivalent subjective intensity map so as to maintain the subjective intensity calculated by the calculation unit. The bidirectional tactile transmission system according to claim 1 or 2.

5. The calculation unit calculates, as the perceptual information, a perceptual intensity determined from the vibration measured by the vibration measurement unit, and The conversion unit converts the signal related to the vibration into a waveform having a different frequency from that of the signal while maintaining the perceived intensity calculated by the calculation unit. The bidirectional tactile transmission system according to claim 1 or 2.

6. The calculation unit divides the signal related to the vibration measured by the vibration measurement unit into predetermined time periods, and calculates the perception intensity for each of the divided predetermined time periods. The bidirectional tactile transmission system according to claim 5 .

7. the conversion unit converts the waveform of a signal in a specific frequency band among the frequency components of the signal by adjusting the perceptual intensity, while converting a signal other than the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the perceptual intensity. The bidirectional tactile transmission system according to claim 5 or 6.

8. the conversion unit converts the waveform of a signal extracted based on a specific signal feature from among the signals by adjusting the perceptual intensity, while converting a signal not extracted by the specific feature into a waveform having a different frequency from that of the signal while maintaining the perceptual intensity. The bidirectional tactile transmission system according to claim 5 or 6.

9. a touch position detection sensor that detects a touch position of a user on a housing of the tactile transmission device; The signal output unit causes the vibrator to output a sense of movement of vibration based on a detection result by the contact position detection sensor. The bidirectional tactile transmission system according to any one of claims 1 to 8.

10. A plurality of the vibration measuring units are provided, The signal output unit causes the vibrator to output a sense of vibration movement based on inputs from each of the plurality of vibration measuring units. The bidirectional tactile transmission system according to any one of claims 1 to 8.

11. the signal output unit outputs sound, video, or light to the other tactile transmission device at the same time as outputting the converted signal from the vibrator; The bidirectional tactile transmission system according to any one of claims 1 to 10.

12. The vibration measuring unit and the vibrator are arranged on a contact plate that is installed on the environment side and transmits vibrations when the user's body comes into contact with the contact plate. The bidirectional tactile transmission system according to any one of claims 1 to 11.

13. The vibration measuring unit and the vibrator are arranged in a wearable device that can be attached to a user's body. The bidirectional tactile transmission system according to any one of claims 1 to 11.

14. The vibration measuring unit and the vibrator are arranged on a device held by a user. The bidirectional tactile transmission system according to any one of claims 1 to 11.

15. The tactile transmission device for use in the bidirectional tactile transmission system according to any one of claims 1 to 14, Among the vibration measurement unit, the calculation unit, the conversion unit, and the signal output unit, the vibration measurement unit may be included in a combination of only the vibration measurement unit, a combination of the vibration measurement unit and the calculation unit, a combination of the vibration measurement unit, the calculation unit, and the conversion unit, or a combination of the vibration measurement unit, the calculation unit, the conversion unit, and the signal output unit. Two-way tactile transmission device.

16. The other tactile transmission device for use in the bidirectional tactile transmission system according to any one of claims 1 to 14, Among the calculation unit, the conversion unit, and the signal output unit, the combination of only the signal output unit, the combination of the conversion unit and the signal output unit, or the combination of the calculation unit, the conversion unit, and the signal output unit is included. Two-way tactile transmission device.

17. A computer in a bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, Measure vibrations generated in the tactile transmission device; Calculating sensory information identified from the measured vibration; converting the vibration-related signal to a predetermined frequency while maintaining the calculated sensory information; The converted signal is output as an output vibration from the vibrator of the other tactile transmission device; Attenuating a specific frequency band by filtering so as to suppress loopback of the vibration output from the vibrator. A two-way tactile transmission program that executes the process.

18. The signal is converted to a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. The bidirectional tactile transmission program according to claim 17, which causes the computer to execute a process.

19. Calculating an envelope determined from the measured vibration as the perception information; While maintaining the calculated envelope, outputting the signal related to the vibration as a bidirectional amplitude modulated signal converted so that the predetermined frequency becomes a carrier frequency; outputting the converted amplitude modulated signal to a vibrator of the other tactile transmission device; The bidirectional tactile transmission program according to claim 17 or 18, which causes the computer to execute a process.

20. Calculating a subjective intensity of the stimulation as the perceptual information from the measured waveform of the vibration; converting the vibration-related signal to the predetermined frequency using a predetermined equivalent subjective intensity map so as to maintain the calculated subjective intensity; The bidirectional tactile transmission program according to claim 17 or 18, which causes the computer to execute a process.

21. Calculating a perception intensity determined from the measured vibration as the perception information; converting the signal relating to the vibration into a waveform having a different frequency from that of the signal while maintaining the calculated perceived intensity; The two-way tactile transmission program according to claim 17 or 18.

22. Dividing the measured signal related to the vibration into predetermined time intervals, and calculating the perceived intensity for each of the divided predetermined time intervals. The bidirectional tactile transmission program according to claim 21 , which causes the computer to execute a process.

23. 23. The bidirectional tactile transmission program according to claim 21 or 22, which causes the computer to execute a process of adjusting the perceptual intensity to convert the waveform of a signal in a specific frequency band among the frequency components of the signal, while converting a signal outside the specific frequency band into a waveform having a different frequency from that of the signal while maintaining the perceptual intensity.

24. a signal extracted based on a specific signal feature among the signals is converted into the waveform by adjusting the perceptual intensity, while a signal not extracted based on the specific feature is converted into a waveform having a different frequency from that of the signal while maintaining the perceptual intensity; The bidirectional tactile transmission program according to claim 21 or 22, which causes the computer to execute a process.

25. A bidirectional tactile transmission method in a bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, comprising: Measure vibrations generated in the tactile transmission device; Calculating sensory information identified from the measured vibration; converting the vibration-related signal to a predetermined frequency while maintaining the calculated sensory information; The converted signal is output as an output vibration from the vibrator of the other tactile transmission device; Attenuating a specific frequency band by filtering so as to suppress loopback of the vibration output from the vibrator. A two-way tactile transmission method for executing a process.

26. The signal is converted to a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. The method of bidirectional tactile transmission according to claim 25.

27. Calculating an envelope determined from the measured vibration as the perception information; While maintaining the calculated envelope, the signal related to the vibration is converted so that the predetermined frequency becomes a carrier frequency, and outputted as an amplitude modulated signal; outputting the converted amplitude modulated signal to a vibrator of the other tactile transmission device; The method for bidirectional tactile transmission according to claim 25 or 26.

28. Calculating a subjective intensity of the stimulation as the perceptual information from the measured waveform of the vibration; converting the vibration-related signal to the predetermined frequency using a predetermined equivalent subjective intensity map so as to maintain the calculated subjective intensity; The method for bidirectional tactile transmission according to claim 25 or 26.

29. Calculating a perception intensity determined from the measured vibration as the perception information; converting the signal relating to the vibration into a waveform having a different frequency from that of the signal while maintaining the calculated perceived intensity; 29. The method for bidirectional tactile transmission according to claim 27 or 28.

30. The measured signal relating to the vibration is divided into predetermined time intervals, and each of the divided predetermined time intervals , calculating the perceived intensity.

30. The method of bidirectional tactile transmission according to claim 29.

31. 31. The method for bidirectional tactile transmission according to claim 29 or 30, wherein the waveform of a signal in a specific frequency band among the frequency components of the signal is converted by adjusting the perceived intensity, while a signal outside the specific frequency band is converted into a waveform having a different frequency from that of the signal while maintaining the perceived intensity.

32. a signal extracted based on a specific signal feature among the signals is converted into the waveform by adjusting the perceptual intensity, while a signal not extracted based on the specific feature is converted into a waveform having a different frequency from that of the signal while maintaining the perceptual intensity; The method for bidirectional tactile transmission according to claim 29 or 30.

33. A bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, a vibration measuring unit that measures vibrations generated in the tactile transmission device; a calculation unit that calculates perceptual information specified from the vibration measured by the vibration measuring unit; a conversion unit that converts the signal related to the vibration into a predetermined frequency while maintaining the sensory information calculated by the calculation unit; a signal output unit that outputs the converted signal from the conversion unit to a vibrator of the other tactile transmission device as an output vibration; Equipped with the conversion unit converts the signal into a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. Two-way tactile transmission system.

34. A computer in a bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, Measure vibrations generated in the tactile transmission device; Calculating sensory information identified from the measured vibration; converting the vibration-related signal to a predetermined frequency while maintaining the calculated sensory information; The converted signal is output as an output vibration from the vibrator of the other tactile transmission device; The signal is converted to a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. A two-way tactile transmission program that executes the process.

35. A bidirectional tactile transmission method in a bidirectional tactile transmission system having a tactile transmission device and another tactile transmission device, comprising: Measure vibrations generated in the tactile transmission device; Calculating sensory information identified from the measured vibration; converting the vibration-related signal to a predetermined frequency while maintaining the calculated sensory information; The converted signal is output as an output vibration from the vibrator of the other tactile transmission device; The signal is converted to a frequency other than a resonant frequency of a housing of the tactile transmission device as the predetermined frequency. A two-way tactile transmission method for executing a process.

Citation Information

Patent Citations

  • Electronic device, signal conversion method and program

    JP2012168775A

  • Haptic driver with attenuation

    JP2020161156A

  • Vibration generator and electronic device

    JP2020201749A

  • Tactile information output device, tactile information generation program, and tactile information communication system

    JP2021051690A

  • Haptic communication device and system for transmitting haptic interaction

    US20170193767A1