Vibration processing device, method, and program

Abstract

A vibration processing device according to an embodiment comprises: a reception unit that receives a vibration sound generated by a vibration motion of an object; and a vibration sound separation unit that separates the vibration sound received by the reception unit into a vibration sound having the frequency characteristic of vibration that can be output by a vibration actuator for reproducing vibration having propagated through the object, and a vibration sound having the frequency characteristic of vibration that can be output by a motion actuator for reproducing the vertical movement of the object.

Description

Vibration processing device, method and program

[0001] FIELD Embodiments of the present invention relate to a vibration processing device, a method, and a program.

[0002] Advances in communication technology have made it possible for people in distant locations to engage in various social activities together via communication. In recent years, advances in virtual reality (VR) technology have made it possible to transmit not only video and audio but also tactile information such as vibrations, enabling remote experiences with a high sense of realism.

[0003] For example, Non-Patent Document 1 discloses a method of using a condenser microphone (sometimes called a microphone) as a tactile sensor to acquire vibration information and acquiring the vibration information as audio information. Furthermore, when reproducing vibrations in a remote location, a vibration actuator is used to convert audio into vibrations and present the vibrations. This allows conventional audio-video transmission means to be used to transmit the vibration information, making it possible to easily transmit the vibration information using existing transmission protocols, etc.

[0004] Non-patent document 1 discloses a method for simultaneously transmitting floor vibration information in addition to audiovisual information from a basketball game being held at a venue, thereby enabling a user to have a highly immersive experience as if they were watching the game on-site, even if they are located far away from the venue.

[0005] Furthermore, if one wishes to experience a competitive sport such as a bicycle race or motorsport remotely, it is expected that a more realistic experience can be created by combining not only vibrations transmitted through the floor as disclosed in Non-Patent Document 1, but also the up and down movement of the vehicle itself.

[0006] The cycle simulator disclosed in Non-Patent Document 2 uses a single-axis motion actuator attached to the front wheel of the simulator to reproduce the acceleration felt by the rider while riding a bicycle, thereby making it possible to present to the rider the inertial forces caused by road surface irregularities, etc.

[0007] Hirohiko Hayakawa et al., "Practice of Live Feeling of Basketball with Haptic Transmission," Proceedings of the 23rd Annual Conference of the Virtual Reality Society of Japan (September 2018). Kei MIYANOUE et al., "An Examination of Reproducibility of Steering and Running Speed ​​on a Cycling Simulator," Human Engineering, Vol. 52, No. 2 (2016).

[0008] When reproducing up and down movement using a single-axis motion actuator or the like as shown in the above-mentioned non-patent document 2, it is common to attach a motion sensor to a bicycle or the like while it is moving, and measure the vertical acceleration of the bicycle to directly measure the perceived acceleration.

[0009] However, it is expected that it would be difficult to attach multiple sensors, such as microphones and motion sensors for measuring vibrations, to a vehicle body used in bicycle racing or motorsports competitions, etc., due to restrictions on the size and weight of the vehicle body. Furthermore, in order to easily transmit vibration information in synchronization with video and audio using an existing system, as in the system disclosed in Non-Patent Document 1, it is desirable to acquire and transmit the vibration information as audio information.

[0010] In other words, it is necessary to measure the vibrations during actual bicycle riding using only a single condenser microphone, and then present vibrations using different vibration actuators from this single vibration information to the remote user.

[0011] The present invention has been made in light of the above circumstances, and its object is to provide a vibration processing device, method, and program that are capable of generating vibrations with a high sense of realism.

[0012] A vibration processing device according to one embodiment of the present invention includes a receiving unit that receives vibration sound generated by the vibration motion of an object, and a vibration sound separation unit that separates the vibration sound received by the receiving unit into vibration sound having frequency characteristics of vibration that can be output by a vibration actuator that reproduces the vibration that has propagated through the object, and vibration sound having frequency characteristics of vibration that can be output by a motion actuator that reproduces the up and down movement of the object.

[0013] A vibration processing method according to one embodiment of the present invention is a method performed by a vibration processing device, and includes receiving, by a receiving unit of the vibration processing device, vibration sound generated by the vibration motion of an object, and separating, by a vibration sound separation unit of the vibration processing device, the vibration sound received by the receiving unit into vibration sound having frequency characteristics of vibration that can be output by a vibration actuator that reproduces the vibration propagated through the object, and vibration sound having frequency characteristics of vibration that can be output by a motion actuator that reproduces the up and down movement of the object.

[0014] According to the present invention, it is possible to generate vibrations that give a high sense of realism.

[0015] FIG. 1 is a block diagram showing an example of the functional configuration of a vibration presentation system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of an application of the transmission side of the vibration presentation system according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of an application of the reception side of the vibration presentation system according to an embodiment of the present invention. FIG. 4 is a block diagram showing an example of the functional configuration of a transmission-side computer. FIG. 5 is a block diagram showing an example of the functional configuration of a reception-side computer. FIG. 6 is a flowchart showing an example of the procedure of processing operations by the transmission-side computer and the reception-side computer. FIG. 7 is a block diagram showing an example of the hardware configuration of a reception-side computer according to an embodiment of the present invention.

[0016] An embodiment of the present invention will be described below with reference to the drawings. The transmitting side of the vibration presentation system according to the embodiment of the present invention includes a vibration sound measurement microphone attached to a vehicle body, such as a bicycle, which is an object that is actually driven outdoors, and capable of collecting vibration sound generated by vibration of the vehicle body while being driven, and a transmitting-side computer that compresses, divides, and transmits the vibration sound in synchronization with surrounding video and audio, such as environmental sounds, during the actual driving.

[0017] In addition, in the above system, the receiving side where the virtual driving using the simulator is performed indoors is equipped with a receiving side computer that receives vibration sounds in synchronization with the transmitted video and audio, a motion actuator that can reproduce the up and down movement of the vehicle body based on the vibration sound information and present it as vibration to the user who is experiencing the virtual driving using the simulator, and a vibration actuator that can convert the vibration sounds into vibrations, thereby reproducing the vibrations propagating through the vehicle body and presenting them to the user.

[0018] In addition, in one embodiment of the present invention, the receiving computer is a vibration processing device that separates the received vibration sound by frequency and generates, from the vibration sound acquired by one microphone, a vibration sound having a relatively low frequency characteristic that can be output by a motion actuator and a vibration sound having a relatively high frequency characteristic that can be output by a vibration actuator.

[0019] By adopting such a configuration, in one embodiment of the present invention, vibration sounds to be used by multiple vibration presentation means can be generated from vibration sounds acquired by a single microphone, making it possible to present vibrations with a more realistic feel.

[0020] FIG. 1 is a block diagram showing an example of the functional configuration of a vibration presentation system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of an application on the transmission side of the vibration presentation system according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of an application on the reception side of the vibration presentation system according to an embodiment of the present invention. As shown in FIGS. 1 to 3 , the vibration presentation system according to this embodiment includes a vehicle body 1, a communication network (NW) 2, and a simulator 3. The simulator 3 is a simulator that allows a user to experience vibrations related to the vehicle body 1. As shown in FIG. 2 , a transmitting computer 11 and a condenser microphone (condenser microphone) 12 are mounted on the vehicle body 1. The condenser microphone 12 collects vibration sounds while the vehicle body 1 is traveling. The transmitting computer 11 encodes and transmits the vibration sounds collected by the condenser microphone 12.

[0021] The condenser microphone 12 may be installed in the vehicle body 1 near the wheel axle or near the point where the wheel contacts the road surface, where vibration sound can be efficiently picked up, but this is not a limitation. The vibration sound transmitted by the transmitting computer 11 is sent to the simulator 3 via the NW2.

[0022] As shown in FIG. 3, the simulator 3 includes a receiving computer 31 , a vibration actuator 32 , and a motion actuator 33 .

[0023] In the example shown in Figures 2 and 3, the condenser microphone 12 is attached to the front wheel side of the vehicle body 1, and the vibration actuator 32 and motion actuator 33 are attached one by one to locations on the housing of the simulator 3 corresponding to the front wheel side of the vehicle body, but they may also be attached to the rear wheel side or rear of the vehicle body so that vibrations at the rear of the vehicle body can be measured and presented.

[0024] Fig. 4 is a block diagram showing an example of the functional configuration of a transmitting computer. Fig. 5 is a block diagram showing an example of the functional configuration of a receiving computer. Fig. 6 is a flowchart showing an example of the procedure of processing operations by the transmitting computer and the receiving computer. In the example shown in Fig. 4, the transmitting computer 11 has a vibration sound acquisition unit 111 and a vibration sound transmission unit 112. The vibration sound acquisition unit 111 acquires vibration sound collected by the condenser microphone 12 while the vehicle body 1 is running, and passes it to the vibration sound transmission unit 112 (S11). The vibration sound transmission unit 112 encodes and transmits the vibration sound passed from the vibration sound acquisition unit 111 (S12).

[0025] In addition, a 360-degree camera or the like may be connected to the transmitting computer 11, and the vibration sound acquisition unit 111 may use this camera to acquire surrounding images and environmental sounds while the vehicle body 1 is moving, and the vibration sound transmission unit 112 may perform synchronization processing between the acquired results and the acquired vibration sound, and then encode and transmit them.

[0026] In the example shown in FIG. 5, the receiving computer 31 includes a vibration sound receiving unit 311 , a vibration sound separating unit 312 , and a vibration sound output unit 313 .

[0027] The vibration sound receiving unit 311 receives and decodes the vibration sound transmitted from the transmitting computer 11 via the NW2, generates vibration sound information from the decoding, and passes it to the vibration sound separating unit 312 (S21).

[0028] The vibration sound separation unit 312 acquires vibration sound information from the vibration sound receiving unit 311 and separates the vibration sound indicated by this information into vibration sound related to the vibration output by the vibration actuator 32 and vibration sound related to the vibration output by the motion actuator 33 (S22).

[0029] The vibration sound output unit 313 outputs the vibration sound associated with the vibration output by the vibration actuator 32, which has been separated by the vibration sound separation unit 312, to the vibration actuator 32. The vibration sound output unit 313 also outputs the vibration sound associated with the vibration output by the motion actuator 33, which has been separated by the vibration sound separation unit 312, to the motion actuator 33 (S23).

[0030] In this embodiment, based on the frequency response of the vibrations that can be output by each of the vibration actuator 32 and the motion actuator 33, the vibration sound separation unit 312 separates the vibration sound into vibrations for each actuator using a band-pass filter or a low-pass filter.

[0031] For example, when the frequency response of the vibration that the vibration actuator 32 can output is 50 to 200 [Hz] and the frequency response of the vibration that the motion actuator 33 can output as up and down movement is 10 to 50 [Hz], the vibration sound separation unit 312 separates the original vibration sound into vibration sounds for each actuator by generating filtered vibration sounds using a band-pass filter or low-pass filter having a cutoff frequency according to each response frequency bandwidth.

[0032] The above separation method for vibration noise is not limited to the above-described example, and for example, a time-frequency mask may be experimentally obtained in advance, and sound source separation means using this time-frequency mask may be used.

[0033] The vibration presentation system according to the present embodiment described above can present different vibrations to multiple actuators having different frequency responses from vibration sounds picked up by a single microphone, thereby enabling the presentation of highly realistic vibrations using vibrations that reproduce the vibrations propagating through the vehicle body and vibrations that reproduce the up and down movement of the vehicle body.

[0034] 7 is a block diagram showing an example of the hardware configuration of a receiving computer according to an embodiment of the present invention. In the example shown in FIG. 7, the receiving computer 31 of the vibration presentation system according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 411A such as a CPU (Central Processing Unit). A program memory 411B, a data memory 412, an input / output interface 413, and a communication interface 414 are connected to this hardware processor 411A via a bus 415. The same applies to the transmitting computer 11 of the vibration presentation system.

[0035] The communication interface 414 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0036] An input device 500 and an output device 600 attached to the receiving computer 31 and used by a user or the like are connected to the input / output interface 413. The input / output interface 413 receives operation data input by a user or the like through the input device 500 such as a keyboard, touch panel, touchpad, or mouse, and outputs output data to the output device 600, which may include a display device using a liquid crystal or organic electroluminescence (EL) display, for display. The input device 500 and the output device 600 may be devices built into the receiving computer 31, or may be input devices and output devices of other information terminals that can communicate with the receiving computer 31 via the network NW.

[0037] The program memory 411B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and stores programs necessary to execute various control processes, etc., according to one embodiment.

[0038] The data memory 412 is a tangible storage medium that is a combination of, for example, the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and is used to store various data acquired and created during various processes.

[0039] The receiving computer 31 according to one embodiment of the present invention may be configured as a data processing device having the units shown in Fig. 1 as software-based processing function units. The storage area used as a work memory or the like by the receiving computer 31 may be configured by using the data memory 412 shown in Fig. 7. However, these configured storage areas are not essential components within the receiving computer 31, and may be areas provided in, for example, an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0040] The processing function unit can be realized by having the hardware processor 411A read and execute a program stored in the program memory 411B. However, the processing function unit may also be realized in various other forms, including an integrated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0041] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), or may be transmitted and distributed via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only executable programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by having the operation controlled by this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0042] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0043] DESCRIPTION OF SYMBOLS 1... Vehicle body 3... Simulator 11... Transmitting computer 12... Condenser microphone 31... Receiving computer 32... Vibration actuator 33... Motion actuator 111... Vibration sound acquisition unit 112... Vibration sound transmission unit 311... Vibration sound reception unit 312... Vibration sound separation unit 313... Vibration sound output unit

Claims

1. A vibration processing device comprising: a receiving unit that receives vibration sound generated by the vibrating motion of an object; and a vibration sound separation unit that separates the vibration sound received by the receiving unit into vibration sound having frequency characteristics of vibration that can be output by a vibration actuator that reproduces vibrations propagated through the object, and vibration sound having frequency characteristics of vibration that can be output by a motion actuator that reproduces the up and down movement of the object.

2. The vibration processing device of claim 1, wherein the vibration sound separation unit separates the vibration sound received by the receiving unit into a vibration sound having a first frequency characteristic that can be output by the vibration actuator, and a vibration sound having a second frequency characteristic that is lower in frequency than the first frequency characteristic that can be output by the motion actuator.

3. The vibration processing device according to claim 2, wherein the vibration sound separation unit separates the vibration sound received by the receiving unit into vibration sound having the first frequency characteristic and vibration sound having the second frequency characteristic by using a filter having a cutoff frequency corresponding to the first frequency characteristic and a filter having a cutoff frequency corresponding to the second frequency characteristic.

4. A method performed by a vibration processing device, comprising: receiving, by a receiving unit of the vibration processing device, vibration sound generated by the vibrating motion of an object; and separating, by a vibration sound separation unit of the vibration processing device, the vibration sound received by the receiving unit into vibration sound having frequency characteristics of vibration that can be output by a vibration actuator that reproduces the vibration propagated through the object, and vibration sound having frequency characteristics of vibration that can be output by a motion actuator that reproduces the up and down movement of the object.

5. A vibration processing program that causes a processor to function as each part of the vibration processing device according to any one of claims 1 to 3.

Images