Vibration processing device, method, and program

Abstract

A vibration processing device according to one embodiment includes: a reception unit that receives a vibration sound generated by vibration of an object; and a vibration sound conversion unit that uses a filter having frequency characteristics of vibration to be propagated in a simulator for reproducing the vibration of the object, to convert the vibration sound received by the reception unit into a vibration sound having the frequency characteristics of vibration that is output by a vibration actuator for reproducing the vibration propagated through the object.

Description

Vibration processing device, method and program

[0001] FIELD Embodiments of the present invention relate to a vibration processing device, a method, and a program.

[0002] Advances in communication technology have made it possible for people in distant locations to engage in various social activities together through communication. In recent years, advances in virtual reality (VR) technology have made it possible to transmit tactile information such as vibrations in addition to video and audio, enabling more realistic remote experiences.

[0003] For example, Non-Patent Document 1 discloses a method of using a condenser microphone (sometimes called a microphone) as a tactile sensor to acquire vibration information and acquire the vibration information as audio information. Furthermore, when reproducing vibrations in a remote location, a vibration actuator is used to convert sound into vibrations and present the vibrations. This allows conventional audio-visual transmission means to be used for transmitting the vibration information, making it possible to easily transmit the vibration information using existing transmission protocols, etc.

[0004] Non-patent document 1 discloses a method for simultaneously transmitting floor vibration information in addition to audiovisual information from a basketball game being held at a venue, thereby enabling a user to have a highly immersive experience as if they were watching the game on-site, even if they are located far away from the venue.

[0005] Hirohiko Hayakawa et al., "Practice of Live Feeling of Basketball with Haptic Transmission," Proceedings of the 23rd Annual Conference of the Virtual Reality Society of Japan (September 2018).

[0006] Consider the case where the realization of a highly realistic experience by transmitting and presenting vibration information, as in the technique disclosed in Non-Patent Document 1, is applied to sports using sports equipment, such as bicycle racing or motor sports.

[0007] For example, when applied to bicycle racing, the bicycle used for actual riding is a racing vehicle, whereas the vehicle used for remote virtual riding is expected to be a vehicle used by a home trainer or the person participating in the experience, and is thought to be made of different materials and have a different shape from the vehicle used for actual riding.

[0008] Therefore, even if the vibration sound obtained during actual driving is transmitted and displayed directly to a remote receiving device, it is expected that the vibrations presented to the user will differ from the actual vibrations, and it is thought that the vibrations do not take into account the material and shape of the vehicle body being used.

[0009] This invention has been made in light of the above circumstances, and its purpose is to provide a vibration processing device, method, and program that are capable of generating vibrations that are suitable for reproduction.

[0010] A vibration processing device according to one aspect of the present invention includes a receiving unit that receives vibration sound generated by vibration of an object, and a vibration sound conversion unit that converts the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of vibration output by a vibration actuator that reproduces the vibration that has propagated through the object, using a filter that has the frequency characteristics of vibration that propagates through a simulator that reproduces the vibration of the object.

[0011] A vibration processing method according to one aspect of the present invention is a method performed by a vibration processing device, and includes receiving vibration sound generated by vibration of an object using a receiving unit of the vibration processing device, and converting the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of vibration output by a vibration actuator that reproduces the vibration propagated through the object using a filter having the frequency characteristics of vibration propagating through a simulator that reproduces the vibration of the object using a vibration sound conversion unit of the vibration processing device.

[0012] According to the present invention, it is possible to generate vibrations suitable for reproduction.

[0013] FIG. 1 is a block diagram showing an example of the functional configuration of a vibration presentation system according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of an application of the transmitting side of the vibration presentation system according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of an application of the receiving side of the vibration presentation system according to an embodiment of the present invention. FIG. 4 is a block diagram showing an example of the functional configuration of a transmitting computer. FIG. 5 is a block diagram showing an example of the functional configuration of a receiving computer. FIG. 6 is a flowchart showing an example of the procedure of processing operations by the transmitting computer and the receiving computer. FIG. 7 is a diagram showing an example of the characteristics of a frequency filter. FIG. 8 is a diagram showing an example of the procedure of generating a frequency filter. FIG. 9 is a block diagram showing an example of the hardware configuration of a receiving computer according to an embodiment of the present invention.

[0014] An embodiment of the present invention will be described below with reference to the drawings. The transmitting side of the vibration presentation system according to the embodiment of the present invention includes a vibration sound measurement microphone attached to a vehicle body, such as a bicycle, which is an object that is actually driven outdoors, and capable of collecting vibration sound generated by vibration of the vehicle body while being driven, and a transmitting-side computer that compresses, divides, and transmits the vibration sound in synchronization with surrounding video and audio, such as environmental sounds, during the actual driving.

[0015] In addition, in the above system, the receiving side where the virtual driving using the simulator is performed indoors is equipped with a receiving computer that receives vibration sound in synchronization with the transmitted video and audio, and a vibration actuator that converts this vibration sound into vibration, thereby reproducing the vibration propagating through the vehicle body and presenting it to the user who is experiencing the virtual driving using the simulator.

[0016] In addition, in one embodiment of the present invention, the receiving computer is a vibration processing device that converts the vibration of the presentation target using a frequency mask of the vibration sound that is generated in advance based on the frequency response characteristics of the receiving vehicle body.

[0017] By adopting such a configuration, in one embodiment of the present invention, vehicle body vibrations generated during actual driving are converted and presented according to the characteristics of the vehicle body used on the receiving side, making it possible to create a more realistic experience.

[0018] Fig. 1 is a block diagram illustrating an example of the functional configuration of a vibration presentation system according to an embodiment of the present invention. Fig. 2 is a diagram illustrating an example of an application of the transmission side of the vibration presentation system according to an embodiment of the present invention.

[0019] 1 to 3, the vibration presentation system according to this embodiment includes a vehicle body 1, a communication network (NW) 2, and a simulator 3. The simulator 3 is a simulator that allows a user to experience vibrations related to the vehicle body 1. The vehicle body 1 is equipped with a transmitting computer 11 and a condenser microphone (condenser microphone) 12. The condenser microphone 12 collects vibration sounds while the vehicle body 1 is running. The transmitting computer 11 encodes and transmits the vibration sounds collected by the condenser microphone 12.

[0020] The location of the condenser microphone 12 on the vehicle body 1 is likely to be near the wheel axle or near the point where the wheel comes into contact with the road surface, where vibration sound can be efficiently picked up, but is not limited to this.

[0021] The vibration sound transmitted by the transmitting computer 11 is sent to the simulator 3 via the NW2.

[0022] The simulator 3 includes a receiving computer 31 and a vibration actuator 32 .

[0023] In the example shown in Figures 2 and 3, the condenser microphone 12 is attached to the front wheel side of the vehicle body 1, and one vibration actuator 32 is attached to the housing of the simulator 3 at a location corresponding to the front wheel side of the vehicle body, but it may also be attached to the rear wheel side or rear of the vehicle body so as to measure and present vibrations at the rear of the vehicle body.

[0024] Fig. 4 is a block diagram showing an example of the functional configuration of a transmitting computer. Fig. 5 is a block diagram showing an example of the functional configuration of a receiving computer. Fig. 6 is a flowchart showing an example of the procedure of processing operations by the transmitting computer and the receiving computer. In the example shown in Fig. 4, the transmitting computer 11 has a vibration sound acquisition unit 111 and a vibration sound transmission unit 112. The vibration sound acquisition unit 111 acquires vibration sound while the vehicle body 1 is running, which is picked up by the condenser microphone 12, and passes it to the vibration sound transmission unit 112 (S11). The vibration sound transmission unit 112 encodes and transmits the acquired vibration sound (S12).

[0025] In addition, a 360-degree camera or the like may be connected to the transmitting computer 11, and the vibration sound acquisition unit 111 may use this camera to acquire surrounding images and environmental sounds while the vehicle body 1 is moving, and the vibration sound transmission unit 112 may perform synchronization processing between the acquired results and the acquired vibration sound, and then encode and transmit them.

[0026] In the example shown in FIG. 5, the receiving computer 31 includes a vibration sound receiving unit 311 , a vibration sound converting unit 312 , a frequency filter storage unit 313 , and a vibration sound output unit 314 .

[0027] The vibration sound receiving unit 311 receives and decodes the vibration sound transmitted from the transmitting computer 11 via the NW2, generates vibration sound information from the decoding, and passes it to the vibration sound converting unit 312 (S21).

[0028] The vibration sound conversion unit 312 uses a frequency filter previously stored in the frequency filter memory unit 313 to convert the vibration sound of the vehicle body 1 during driving, which is indicated by the vibration sound information, to match the vibration characteristics of the housing of the simulator 3 (S22).

[0029] 7 is a diagram showing an example of the characteristics of a frequency filter. As shown in Fig. 7, the characteristics of a frequency filter are shown as a graph with the horizontal axis representing frequency and the vertical axis representing the gain at that frequency, i.e., the ratio of input to output.

[0030] Next, an example of a method for generating the frequency filter stored in the frequency filter storage unit 313 will be described. FIG. 8 is a diagram showing an example of a frequency filter generation procedure. The frequency filter according to the embodiment takes into consideration both the difference between the frequency characteristics of the vibration of the vehicle body 1 and the frequency characteristics of the vibration of the housing of the simulator 3, and frequency selection for blocking sounds above a specific frequency that the user cannot perceive as vibration. First, the frequency characteristics of the known vibration of the vehicle body 1 and the known vibration of the housing of the simulator 3 are measured in advance, and the rate of increase or decrease in the power ratio for each frequency between the vibrations is calculated, i.e., the rate of increase or decrease in gain (symbol c in FIG. 8 ) between the frequency characteristics of the vibration of the vehicle body 1 (symbol a in FIG. 8 ) and the frequency characteristics of the vibration of the housing of the simulator 3 (symbol b in FIG. 8 ). Then, this increase / decrease rate is multiplied by a low-pass filter (symbol d in FIG. 8) having a frequency characteristic of attenuating the gain of frequencies higher than a certain cutoff frequency, and a frequency filter (symbol e in FIG. 8) having the increase / decrease rate of the gain of the frequency band to be output as vibration noise and the characteristic of attenuating the vibration noise of frequencies higher than the cutoff frequency, i.e., the frequency filter shown in FIG. 7, is obtained by pre-measurement.

[0031] In this way, a frequency filter can be generated that takes into account the characteristics of each vehicle body.

[0032] The vibration sound converted by the vibration sound converting unit 312 in S22 is output as vibration by the vibration sound output unit 314 via the vibration actuator 32.

[0033] That is, the vibration sound conversion unit 312 converts the vibration sound received by the vibration sound receiving unit 311 into vibration sound having the frequency characteristics of the vibration output by the vibration actuator 32 using a low-pass filter having a cutoff frequency according to the difference between the frequency characteristics of the vibration propagated through the vehicle body 1 and the frequency characteristics of the vibration propagating through the housing of the simulator 3, as well as the frequency characteristics to be treated as the vibration to be presented to the user using the simulator 3.

[0034] The vibration sound conversion unit 312 converts the vibration sound received by the vibration sound receiving unit 311 into vibration sound having the frequency characteristics of the vibration to be output from the vibration actuator 32 using a filter having a cutoff frequency according to the frequency characteristics of the vibration that propagates through the housing of the simulator 3, which is formed by multiplying a filter having a cutoff frequency according to the frequency characteristics of the difference in vibration intensity based on the difference between the frequency characteristics of the vibration that propagates through the vehicle body 1 and the frequency characteristics of the vibration that propagates through the housing of the simulator 3, and the frequency characteristics that are treated as the vibration.

[0035] The vibration presentation system according to the present embodiment, as explained above, presents vibrations that take into consideration the characteristics of the simulator used by the participant, based on the vibration sounds of the vehicle body during driving. Therefore, the participant using the simulator 3 can experience more realistic vehicle body vibrations while using the vehicle body of the simulator that they are actually using.

[0036] 9 is a block diagram showing an example of the hardware configuration of a receiving computer according to an embodiment of the present invention. In the example shown in FIG. 9, the receiving computer 31 of the vibration presentation system according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 411A such as a CPU (Central Processing Unit). A program memory 411B, a data memory 412, an input / output interface 413, and a communication interface 414 are connected to this hardware processor 411A via a bus 415. The same applies to the transmitting computer 11 of the vibration presentation system.

[0037] The communication interface 414 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from the communication network NW. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0038] An input device 500 and an output device 600 attached to the receiving computer 31 and used by a user or the like are connected to the input / output interface 413. The input / output interface 413 can take in operation data input by a user or the like through the input device 500 such as a keyboard, touch panel, touchpad, or mouse, and can output output data to an output device 600 including a display device using a liquid crystal or organic electroluminescence (EL) display, for display. The input device 500 and the output device 600 may be devices built into the receiving computer 31, or may be input devices and output devices of other information terminals that can communicate with the receiving computer 31 via the network NW.

[0039] The program memory 411B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a ROM, and can store programs necessary to execute various control processes, etc., according to one embodiment.

[0040] The data memory 412 is a tangible storage medium, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM, and can be used to store various data or information acquired and created during various processes.

[0041] The receiving computer 31 according to one embodiment of the present invention can be configured as a data processing device having the respective units shown in FIG. 1 as software-based processing function units.

[0042] The information storage unit used as a work memory or the like by each unit of the receiving computer 31 can be configured by using the data memory 412 shown in Fig. 9. However, these configured storage areas are not essential components within the receiving computer 31, and may be areas provided in an external storage medium such as a USB (Universal Serial Bus) memory, or in a storage device such as a database server located in the cloud.

[0043] The processing function units in each of the above-described units can be realized by reading and executing a program stored in the program memory 411B by the hardware processor 411A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0044] The methods described in each embodiment may be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), or may be transmitted and distributed via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only executable programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-described processing by having the operation controlled by this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0045] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0046] REFERENCE SIGNS LIST 1 vehicle body 3 simulator 11 transmitting computer 12 condenser microphone 31 receiving computer 32 vibration actuator 111 vibration sound acquisition unit 112 vibration sound transmission unit 311 vibration sound reception unit 312 vibration sound conversion unit 313 frequency filter storage unit 314 vibration sound output unit

Claims

1. A vibration processing device comprising: a receiving unit that receives vibration sound generated by vibration of an object; and a vibration sound conversion unit that converts the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of a vibration output by a vibration actuator that reproduces the vibration propagated through the object, using a filter having the frequency characteristics of a vibration propagating through a simulator that reproduces the vibration of the object.

2. The vibration processing device described in claim 1, wherein the vibration sound conversion unit converts the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of the vibration output by the vibration actuator using a filter having the frequency characteristics of the vibration propagating through the simulator, based on a filter having a cutoff frequency corresponding to the difference between the frequency characteristics of the vibration propagated through the object and the frequency characteristics of the vibration propagating through the simulator, and corresponding to the frequency characteristics to be treated as the vibration to be presented to a user using the simulator.

3. The vibration sound conversion unit of claim 2 converts the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of the vibration output by the vibration actuator using a filter having a cutoff frequency according to the frequency characteristics of the vibration propagating through the simulator, the filter being obtained by multiplying a filter having a cutoff frequency according to the frequency characteristics of the difference in vibration intensity based on the difference between the frequency characteristics of the vibration propagated through the object and the frequency characteristics of the vibration propagating through the simulator, and the frequency characteristics to be treated as the vibration.

4. A method performed by a vibration processing device, comprising: receiving, by a receiving unit of the vibration processing device, vibration sound generated by vibration of an object; and converting, by a vibration sound conversion unit of the vibration processing device, the vibration sound received by the receiving unit into vibration sound having the frequency characteristics of vibration output by a vibration actuator that reproduces the vibration propagated through the object, using a filter having the frequency characteristics of vibration propagating through a simulator that reproduces the vibration of the object.

5. A vibration processing program that causes a processor to function as each part of the vibration processing device according to any one of claims 1 to 3.

Images