Information processing device, information processing method, and information processing program

Abstract

To provide an information processing device, information processing method, and information processing program capable of suppressing reduction in immersive feeling of music and images caused by vibration for notifying information.SOLUTION: An information processing device according to the embodiment is an information processing device that outputs acoustic sounds based on content and immersive vibrations. The information processing device detects notification of information via vibration. The information processing device increases a volume level of a specific frequency band in the acoustic sounds of the content during notification of information via vibration.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The disclosed embodiments relate to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] There are systems for notifying users (drivers) of information by vibration in automobiles, etc. There are also AV (Audio Visual) systems that apply vibrations synchronized with music and images to users in order to enhance the sense of realism of the music and images (see, for example, Patent Document 1).

[0003] In such a system, notification of information is generally given priority, so when notification of information is required while music and video are being played, the vibrations synchronized with the music and video are stopped, or the vibrations synchronized with the music and video are mixed with the vibrations for notifying information. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-003933 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional systems, when vibrations to notify information are generated while music and video are being played, the sense of realism of the music and video is reduced by stopping the vibrations synchronized with the music and video, or by mixing the vibrations synchronized with the music and video with the vibrations to notify information.

[0006] One aspect of the embodiment has been made in consideration of the above, and aims to provide an information processing device, an information processing method, and an information processing program that can suppress a decrease in the realism of music and images due to vibrations used to notify information. [Means for solving the problem]

[0007] According to one aspect of the embodiment, an information processing device outputs audio based on content and realistic vibrations. The information processing device detects notification of information via vibrations. When notifying the information via vibrations, the information processing device increases a volume level of a specific frequency band of the audio based on the content. [Effects of the Invention]

[0008] The information processing device, information processing method, and information processing program according to the embodiments increase the volume level of a specific frequency band in the acoustic sound of the content when notifying information by vibration, thereby suppressing a decrease in the sense of realism caused by vibration when notifying information. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an information processing device according to an embodiment. [Figure 2] FIG. 2 is an explanatory diagram showing the installation positions of the first to fourth vibrators according to the embodiment. [Figure 3] FIG. 3 is a diagram showing the configuration of the signal processing unit. [Figure 4] FIG. 4 is an explanatory diagram of a method for notifying the direction of an object relative to the vehicle by vibration. [Figure 5] FIG. 5 is an explanatory diagram of a method for notifying the direction of an object relative to the vehicle by vibration. [Figure 6] FIG. 6 is an explanatory diagram of a method for notifying the direction of an object relative to the vehicle by vibration. [Figure 7] FIG. 7 is an explanatory diagram of a method for producing video content using vibration. [Figure 8] FIG. 8 is an explanatory diagram of fading in and fading out of vibration according to the embodiment. [Figure 9] FIG. 9 is a flowchart illustrating an example of processing executed by the controller according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an information processing device, an information processing method, and an information processing program will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below. The information processing device according to the embodiment is mounted on a vehicle, plays music and video, and has a function of notifying a user (driver) by vibration when an object (hereinafter simply referred to as an "object") that the user (driver) should pay attention to is detected around the vehicle. The object is, for example, another vehicle or a pedestrian approaching the vehicle.

[0011] In the following description, the direction of straight-ahead travel of the vehicle, from the driver's seat toward the steering wheel, is referred to as the "forward direction." Furthermore, the direction of straight-ahead travel of the vehicle, from the steering wheel toward the driver's seat, is referred to as the "rearward direction." Furthermore, the direction perpendicular to the straight-ahead travel of the vehicle and a vertical line, from the right side toward the left side of the driver facing forward, is referred to as the "leftward direction." Furthermore, the direction perpendicular to the straight-ahead travel of the vehicle and a vertical line, from the left side toward the right side of the driver facing forward, is referred to as the "rightward direction." Note that these directions are simply names used for the purpose of explanation and are not intended to limit the actual positional relationships and directions.

[0012] [1. Configuration of information processing device] Fig. 1 is an explanatory diagram showing an example of the configuration of an information processing device 1 according to an embodiment. As shown in Fig. 1, the information processing device 1 is connected to a surroundings monitoring device 11, an operation device 12, a display device 13, a sound output device 14, and first to fourth vibrators 21 to 24, and constitutes an in-vehicle information system SYS. The devices of the in-vehicle information system SYS are connected to each other via wired or wireless communication, etc., and are configured to be able to operate in cooperation with each other.

[0013] The periphery monitoring device 11 is a device that detects objects present around the vehicle. The periphery monitoring device 11 includes, for example, a radar, a LiDAR, an on-board camera, and a detection processing unit 11a. The detection processing unit 11a is configured with a microcomputer or the like, and determines the presence or absence of objects present around the vehicle and the direction of the objects relative to the vehicle based on the detection results from the radar and LiDAR.

[0014] Furthermore, the detection processing unit 11a performs image recognition on the video of the surroundings of the vehicle captured by the on-board camera to determine whether or not there is an object around the vehicle, the direction of the object relative to the vehicle, etc. The surroundings monitoring device 11 (detection processing unit 11a) outputs information indicating whether or not there is an object around the vehicle, the direction of the object relative to the vehicle, etc. to the information processing device 1.

[0015] The operation device 12 includes various operation buttons operated by the user. The operation device 12 may be a touch panel display that accepts user operations. The user can use the operation device 12 to perform various operations on the information processing device 1, the periphery monitoring device 11, etc., i.e., the in-vehicle information system SYS, such as operations related to music and video playback. In the in-vehicle information system SYS of this example, the operation device 12 outputs an operation signal corresponding to the user's operation to the information processing device 1 (which is acquired by the controller 2 of the information processing device 1), and the controller 2 transmits it to each part of the in-vehicle information system SYS, such as the periphery monitoring device 11.

[0016] The display device 13 is a device that displays various information, images, etc. under the control of the information processing device 1, and is configured with a liquid crystal display or the like. The sound output device 14 is a device that outputs various information, music, etc. under the control of the information processing device 1, and is configured with an electromagnetic speaker or the like. The first to fourth vibrators 21 to 24 are vibrators that vibrate under the control of the information processing device 1, and are configured with an electromagnetically driven exciter or the like.

[0017] The information processing device 1 also includes a controller 2 that controls various operations of the information processing device 1. The controller 2 includes a control unit 3 and a storage unit 4. The storage unit 4 is, for example, an information storage device such as a data flash, and stores various data such as music / video information 41 and an information processing program 42. Note that the storage unit 4 also stores various data other than the programs executed by the control unit 3 (such as the information processing program 42), and therefore it can be said that a part of the data constitutes the controller 2.

[0018] The music video information 41 includes audio information of multiple music contents. The music video information 41 also includes video information and audio information of multiple video contents. The music video information 41 can be information read from a storage medium such as an optical disc or information distributed via a communication network. The information processing program 42 is a program executed by the control unit 3 to control the display device 13, the sound output device 14, and the first to fourth vibrators 21 to 24. The information processing program 42 is often incorporated (stored) into the information processing device 1 during manufacturing, but may also be stored in a storage device from the outside via a communication line, etc., after manufacturing of the information processing device 1 or while the information processing device 1 is in use (updated to a new version of the program).

[0019] The control unit 3 includes a microcomputer having a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and various circuits. The control unit 3 includes a vibration control unit 31, a display control unit 32, and a sound control unit 33, which function by reading an information processing program 42 from the storage unit 4 and executing it using the RAM as a working area.

[0020] The vibration control unit 31, the display control unit 32, and the sound control unit 33 included in the control unit 3 may be partially or entirely configured using hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0021] The vibration control unit 31, the display control unit 32, and the audio control unit 33 included in the control unit 3 each realize or execute the information processing functions described below. Note that the internal configuration of the control unit 3 is not limited to the configuration shown in Fig. 1, and may be any other configuration as long as it performs the information processing described below.

[0022] When an operation to play video content is performed on the operation device 12, the display control unit 32 reads out video information of the video content selected by the user in response to the operation from the storage unit 4, and displays and plays it on the display device 13. When an operation to play video content is performed on the operation device 12, the audio control unit 33 reads out acoustic information of the video content selected by the user in response to the operation from the storage unit 4, and plays it by emitting sound from the sound output device 14. Furthermore, the display control unit 32 displays and plays out a corresponding image (for example, an image suggesting the direction in which an obstacle exists) on the display device 13 based on image information and a display instruction signal from each device of the in-vehicle information system SYS, for example, the periphery monitoring device 11.

[0023] Furthermore, when an operation to play music content is performed on the operation device 12, the audio control unit 33 reads out acoustic information of the music content selected by the user in response to the operation from the storage unit 4, and reproduces the same by outputting sound from the sound output device 14. When an operation to play music content is performed on the operation device 12, the audio control unit 33 reads out acoustic information of the music content selected by the user in response to the operation from the storage unit 4, and reproduces the same by outputting sound from the sound output device 14. Furthermore, the audio control unit 33 reproduces a corresponding sound (for example, a sound indicating the direction of an obstacle) from the sound output device 14 based on audio information and audio instruction signals from each device of the in-vehicle information system SYS, for example, the periphery monitoring device 11.

[0024] For ease of understanding, a signal based on audio information such as music content will be referred to as an audio signal, a signal based on a notification from the periphery monitoring device 11 will be referred to as a notification sound signal, and the output sounds from each signal will be referred to as an audio sound and a notification sound. Furthermore, to distinguish between the original sound and the processed (output) sound, the signals will be referred to as an original audio signal, an output audio signal, an original notification sound signal, and an output notification sound signal, and also as an original audio sound, an output audio sound, an original notification sound, and an output notification sound.

[0025] The vibration control unit 31 controls the first to fourth vibrators 21 to 24. When the audio control unit 33 reproduces an audio signal of music content, the vibration control unit 31 vibrates the vibrators 21 to 24 with a vibration signal corresponding to the audio signal of the music content to enhance the sense of realism (hereinafter referred to as a realistic vibration signal, and vibrations generated based on the realistic vibration signal are referred to as realistic vibrations). The vibration signal corresponding to the audio signal is a signal for emitting vibrations that enhance the sense of realism in coordination with the audio reproduction, and may be, for example, a signal in which the level of a signal in a frequency range in the audio signal to which humans have high vibration sensitivity is adjusted (for example, a signal in which the signal level of a predetermined low-frequency band in the audio signal is increased), a signal predetermined in accordance with the frequency characteristics of the audio signal (a signal with a frequency that has a predetermined relationship with the peak frequency of the audio signal (for example, 1 / 2 the frequency)), or the like.

[0026] Furthermore, when video content is played by the display control unit 32 and an audio signal based on audio information of the video content is played by the audio control unit 33, the vibration control unit 31 vibrates the vibrators 21 to 24 with a realistic vibration signal corresponding to the audio signal of the video content. Furthermore, when a notification target object is detected by the periphery monitoring device 11, the vibration control unit 31 vibrates at least one of the vibrators 21 to 24 with a vibration signal for notifying information indicating that the object is detected (hereinafter referred to as a notification vibration signal, and the vibration generated based on the notification vibration signal is referred to as a notification vibration).

[0027] [2. Transducer installation position] Next, the installation positions of the first to fourth vibrators 21 to 24 will be described with reference to Fig. 2. Fig. 2 is an explanatory diagram showing the installation positions of the first to fourth vibrators 21 to 24 according to the embodiment, and is also a structural diagram showing the structure of the seat 5 on which the first to fourth vibrators 21 to 24 are installed.

[0028] In Fig. 2, arrow X indicates the forward direction, arrow Y indicates the rightward direction, and arrow Z indicates a direction parallel to the upward direction. As shown in Fig. 2, the first to fourth oscillators 21 to 24 are provided on a seat 5. The seat 5 has a seat portion 51 and a backrest portion 52. The seat portion 51 supports the buttocks and thighs of an occupant sitting on the seat 5. The backrest portion 52 supports the back of the occupant sitting on the seat 5.

[0029] The first to fourth vibrators 21 to 24 are arranged on the seat 51. Specifically, the first to fourth vibrators 21 to 24 are embedded inside the seat 51. The first to fourth vibrators 21 to 24 may be arranged on the surface of the seat 51. The first to fourth vibrators 21 to 24 may be attached to the upper surface of the seat 51.

[0030] The first vibrator 21 is arranged on the front left side of the seat 51. The second vibrator 22 is arranged on the front right side of the seat 51. The third vibrator 23 is arranged on the rear left side of the seat 51. The fourth vibrator 24 is arranged on the rear right side of the seat 51. The number and positions of the vibrators arranged on the seat 51 may be changed as appropriate from the configuration of this embodiment.

[0031] The first to fourth vibrators 21 to 24 are arranged so that vibrations are sufficiently transmitted to the occupant sitting in the seat 5. The first to fourth vibrators 21 to 24 are arranged so that vibrations are sufficiently transmitted to the body of the occupant sitting on the seat portion 51. For example, the first to fourth vibrators 21 to 24 are arranged so as to be located below the left front, right front, left rear, and right rear parts of the buttocks of the occupant sitting in the seat 5.

[0032] The first to fourth vibrators 21 to 24 are vibrators of the same type (same configuration). The first to fourth vibrators 21 to 24 may be vibrators with an electric / magnetic circuit configuration in which a speaker diaphragm (structure suitable for acoustic conversion) is replaced with a diaphragm suitable for vibration transmission, or vibrators with a configuration using a piezoelectric element. The information processing device 1 is configured to be able to individually control each of the first to fourth vibrators 21 to 24. In other words, the information processing device 1 can individually control the start and stop of vibration and the vibration amount of each of the first to fourth vibrators 21 to 24.

[0033] [3. Control of the vibrator and sound output device by the controller] The process of generating the acoustic signal, the notification sound signal, the realistic vibration signal, and the notification vibration will be described with reference to Fig. 3. Fig. 3 is a configuration diagram showing the configuration of the signal processing unit 7.

[0034] The signal processing unit 7 is composed of an audio signal processing unit 70, a notification signal processing unit 80, and mixers 75 and 76. The audio signal processing unit 70 processes the original audio signal to generate an output audio signal and a realistic vibration signal. The notification signal processing unit 80 processes the original notification sound signal to generate an output notification sound signal and a notification vibration signal. The mixer 75 mixes the output audio signal and the output notification sound signal and outputs the result to the sound output device 14. The mixer 75 also mixes the realistic vibration signal and the notification vibration signal and outputs the result to the vibrator. Note that each component of the signal processing unit 7 is provided according to the number of channels (number of speakers) of the sound output device 14 and the number of channels (number of vibrators) of the vibrator, but is omitted here because the configurations are similar.

[0035] The acoustic signal processing unit 70 and the notification signal processing unit 80 perform signal processing in response to control signals from the controller 2. First, the process of generating an output acoustic signal will be described. Note that the generation of an output acoustic signal also includes normal sound field effect processing, such as surround sound generation processing, but here we will describe signal processing related to vibration.

[0036] The graphic equalizer 71 in the acoustic signal processing unit 70 performs processing to change the frequency characteristics of the original acoustic signal based on a control signal from the controller 2. That is, the graphic equalizer 71 increases or decreases the signal level of a specific frequency band of the original acoustic signal based on a control signal from the controller 2. The level adjuster 72 then adjusts (increases or decreases the signal level) the level (all frequency bands) of the original acoustic signal whose frequency characteristics have been adjusted by the graphic equalizer 71, and generates an output acoustic signal. This makes it possible to adjust the frequency characteristics of the original acoustic signal, fade in, fade out, etc.

[0037] The low-pass filter 73 in the acoustic signal processing unit 70 performs processing to extract a signal in a frequency band (low frequency band: for example, an appropriate band is set based on experiments, etc.) suitable for enhancing the sense of realism from the original acoustic signal. Furthermore, instead of or in addition to the low-pass filter 73, signal processing using a frequency shift filter or the like may be used. The level adjuster 74 then adjusts (increases or decreases the signal level) the level of the signal in the frequency band extracted by the low-pass filter 73 so that it becomes a level suitable for the realistic vibration, thereby generating a realistic vibration signal. This results in the generation of a realistic vibration signal with appropriate characteristics and level based on the original acoustic signal.

[0038] The notification vibration source unit 81 in the notification signal processing unit 80 outputs a notification vibration signal (referred to as an original notification vibration signal) that is the source of the notification vibration signal, and is configured with a memory in which each notification vibration signal corresponding to the notification content is stored. The notification vibration selection unit 82 selects an original notification vibration signal to be processed from the notification vibration source unit 81 according to the notification content. The level adjuster 83 then adjusts (increases or decreases the signal level) the level of the original notification vibration signal selected by the notification vibration selection unit 82 to a level appropriate for the notification vibration, thereby generating a notification vibration signal. For example, when notifying the presence of an obstacle around the vehicle, an original notification vibration signal for notifying the presence of the obstacle is selected from the notification vibration source unit 81 based on an information notification command from the controller 2 (a command to notify the presence of an obstacle around the vehicle), and the level of the original notification vibration signal is adjusted, and the notification vibration signal for notifying the presence of the obstacle is output from the level adjuster 83. Note that the original notification vibration signal can also be generated by other methods, such as acquiring ambient sound using a microphone or the like and processing the acquired sound to generate the original notification vibration signal.

[0039] The notification sound source unit 84 in the notification signal processing unit 80 outputs a notification sound signal (referred to as an original notification sound signal) that is the source of the notification sound signal, and is configured with a memory in which each notification sound signal corresponding to the notification content is stored. The notification sound selection unit 85 selects an original notification sound signal to be processed from the notification sound source unit 84 depending on the notification content. The level adjuster 86 then adjusts (increases or decreases the signal level) the level of the original notification sound signal selected by the notification sound selection unit 85 to a level appropriate for the notification sound, thereby generating a notification sound signal. For example, when notifying the presence of an obstacle around the vehicle, an original notification sound signal for notifying the presence of the obstacle is selected from the notification sound source unit 84 based on an information notification command from the controller 2 (a command to notify the presence of an obstacle around the vehicle), and the level of the original notification sound signal is adjusted so that the notification sound signal for notifying the presence of the obstacle is output from the level adjuster 86. Note that the original notification sound signal can also be generated by other methods, such as acquiring ambient sound using a microphone or the like and processing the acquired sound to generate the original notification sound signal.

[0040] The mixer 75 mixes the output acoustic signal from the acoustic signal processing unit 70 with the notification sound signal from the notification signal processing unit 80, and outputs the result to the sound output device 14. The mixer 76 mixes the realistic vibration signal from the acoustic signal processing unit 70 with the notification vibration signal from the notification signal processing unit 80, and outputs the result to the vibrator.

[0041] Each component of the signal processing unit 7 can be realized by an acoustic circuit made up of various acoustic components, or by digital signal processing using a digital signal processor or the like.

[0042] When an object is detected during playback of music content or video content and notification of this is given by vibration, the controller 2 controls the vibrators 21-24 and the sound output device 14 to prevent a decrease in the sense of realism of the music content or video content. In other words, in such a case, the controller 2 controls the signal processing unit 7 to perform processing so that an appropriate acoustic sound, notification sound, realistic vibration, and notification vibration are provided to the user.

[0043] Furthermore, when an object is detected during music or video playback and the controller 2 notifies the user of this by vibration, the controller 2 controls the vibrators 21 to 24 so that the difference between the vibration for sound and the vibration for notification is clear. In other words, in such a case, the controller 2 controls the signal processing unit 7 to perform processing so that appropriate realistic vibration and notification vibration are provided to the user.

[0044] [3-1. Example of controlling a sound output device] When the controller 2 vibrates the vibrators 21-24 with a notification vibration signal for notifying information during a period when the vibrators 21-24 are vibrated with a realistic vibration signal corresponding to the sound of music and video, the realistic vibration signal is deformed (the realistic vibration is reduced to prioritize notification by the notification vibration) or the realistic vibration is reduced due to interference of the notification vibration signal. Therefore, to compensate for such a reduction in the realistic feeling, the volume of a specific frequency band of the acoustic sound (a frequency band that can compensate for the realistic feeling improvement effect due to vibration) is increased. In other words, a process is performed to increase the signal level of the specific frequency band of the acoustic signal.

[0045] Low-frequency sounds in acoustic sounds can be sensed not only by the ears but also by the skin, and have the effect of making the user feel vibrations. Therefore, in this embodiment, low-frequency sounds in acoustic sounds are added and enhanced to compensate for the reduction in realism due to the realism vibration by enhancing the low-frequency sounds in acoustic sounds. Furthermore, it is believed that the reduction in realism due to the realism vibration is significantly affected by the realism vibration in the frequency band of the notification vibration. Therefore, the level of the sound in the frequency band of the notification vibration in acoustic sounds is increased to more appropriately compensate for the reduction in realism. Note that, in order to improve the distinguishability of the notification vibration, the realism vibration (all bands, the frequency band of the notification vibration, etc.) may be stopped or the level reduced when the notification vibration is output.

[0046] Specifically, the controller 2 increases the signal in the frequency band of the notification vibration in the audio signal. As a result, even if the frequency band of the notification vibration in the realistic vibration is masked by the notification vibration and becomes difficult for the user to recognize, the controller 2 increases the volume of the frequency band in the audio sound, thereby compensating for the vibration sensation in the frequency band and suppressing a decrease in the realistic sensation due to the notification vibration in music and video playback.

[0047] The frequency band that humans can perceive particularly well through their sense of touch is the band from 50 Hz to 150 Hz, and therefore this frequency band is also used for the notification vibration and the realistic vibration. Therefore, when the controller 2 generates the notification vibration during a period in which realistic vibration is being output during music and video playback, the controller 2 amplifies the frequency band from 50 Hz to 150 Hz in the audio signal using the graphic equalizer 71 to increase the volume of the audio in that frequency band. This makes it possible to compensate for the realistic sensation caused by the realistic vibration, which may be reduced by the notification vibration, by increasing the volume of the audio in that frequency band.

[0048] Furthermore, when the controller 2 starts vibration in the notification vibration frequency band during playback of music content and video content, the controller 2 gradually increases the volume of the acoustic sound in a specific frequency band to compensate for the sense of realism caused by the realistic vibration to a predetermined level. The predetermined level is a volume level that can compensate for the reduction in the sense of realism caused by the vibration for information notification, and is a level determined by a designer or developer through experiments or the like.

[0049] Thereafter, when ending vibration in the notification vibration frequency band, the controller 2 gradually reduces the volume of the acoustic sound in the specific frequency band for compensating for the sense of realism due to the realism vibration to the volume of normal playback, thereby reducing the sense of discomfort caused by a sudden change in the playback volume of music, etc. when suppressing the decrease in the sense of realism due to the realism vibration.

[0050] [3-2. Control of oscillators] Furthermore, when the occurrence periods of the realistic vibration and the notification vibration overlap, the controller 2 differentiates the frequency bandwidths of the realistic vibration and the notification vibration. This difference in vibration frequency bands improves the distinguishability between the realistic vibration and the notification vibration, and even if a notification by vibration occurs while viewing content, the notification can be easily recognized.

[0051] Specifically, when notifying by vibration, the controller 2 vibrates at least one of the first to fourth vibrators 21 to 24 using a notification vibration at a frequency outside the frequency band of the realistic vibration, preferably a frequency band that is a predetermined frequency higher than the frequency band. Note that the predetermined frequency is a frequency at which the difference in vibration frequency can be recognized, and is determined, for example, by a designer or developer through experiments. For example, the realistic vibration frequency band is a frequency band from 50 Hz to 150 Hz that is effective in improving the sense of realism, and the notification vibration frequency band, which does not require a sense of realism, is the upper frequency band of 200 Hz to 300 Hz.

[0052] This allows the frequency band of the realistic vibration to be a frequency band that is effective in improving the sense of realism, so that the realistic vibration can be effectively used to improve the sense of realism.

[0053] Furthermore, the controller 2 narrows the frequency band of the notification vibration compared to the frequency band of the realistic vibration. The narrower the bandwidth of the vibration frequency is to some extent, the easier it is for the user to recognize the presence or absence of vibration. Specifically, the controller 2 vibrates the first to fourth vibrators 21 to 24 with notification vibration of a frequency band narrower than the frequency band of the realistic vibration, for example, notification vibration of a fixed frequency (bandwidth is essentially 0). This can further increase the distinguishability of the notification vibration from the realistic vibration.

[0054] Furthermore, when notifying information, there is little need for vibrations in the low frequency range that brings about a sense of impact and the high frequency range that brings about sharpness. For this reason, the controller 2 uses the band from 30 Hz to 200 Hz for the realistic vibration, and the band from 70 Hz to 150 Hz, excluding the low and high frequencies of the band used for the realistic vibration, for the notification vibration. This makes it possible to reduce interference between the notification vibration and the realistic vibration while maintaining the recognizability of the notification via the notification vibration.

[0055] Furthermore, the controller 2 uses the first to fourth vibrators 21 to 24 depending on the content of the information to be notified and the output status of the realistic vibration signal. Specifically, the controller 2 preferentially selects a vibrator to be used for notification vibration among the first to fourth vibrators 21 to 24, and uses the remaining vibrators for realistic vibration.

[0056] When one vibrator is vibrated, the vibration position (vibration localization point) that a person intuitively grasps (feels) is the position of that vibrator. When two vibrators are vibrated, it can be understood that the vibration localization point is located on the line connecting the two vibrators, and the distance from each vibrator to the vibration localization point is inversely proportional to the strength of each vibrator. For example, if the vibration strength of the vibrator at coordinates (0,0) is 1 and the vibration strength of the vibrator at coordinates (3,0) is 2, the vibration localization point will be (2,0). Using this characteristic, by vibrating each vibrator set in seat 5, it is possible to add a directional component to the information. For example, by setting the vibration localization point to the right front of seat 5, it is possible to notify the user by vibration that there is an obstacle (another vehicle) to the right front of the vehicle. Figures 4 to 6 are explanatory diagrams of a method of notifying the user of the direction of an object relative to the vehicle by vibration. Figure 7 is an explanatory diagram of a method of producing video content using vibration.

[0057] 4, when another vehicle 61 (object) traveling to the right front of the host vehicle 6 (user) is detected by the periphery monitoring device 11, the controller 2 notifies the presence and direction of the other vehicle 61 by vibration using the second oscillator 22 and the fourth oscillator 24 for information notification. Note that when the direction of the other vehicle 61 is like this, the first oscillator 21 and the third oscillator 23 are not used (unnecessary).

[0058] At this time, the controller 2 increases the vibration amount of the second vibrator 22 and decreases the vibration amount of the fourth vibrator 24 according to the direction (right front) of the other vehicle 61. As a result, the controller 2 matches the direction from the seating center P1 of the person sitting in the seat 5 (the center of gravity P1 of the area A surrounded by the first to fourth vibrators 21 to 24) toward the vibration localization point P2 generated by the vibrations of the second vibrator 22 and the fourth vibrator 24 with the direction of the other vehicle 61 relative to the user's vehicle 6. Therefore, the user (the person sitting in the seat 5) can grasp the direction of the other vehicle 61 relative to the user's vehicle 6.

[0059] At this time, the realistic vibration during playback of the content is output using the unused first oscillator 21 and third oscillator 23. For example, as shown in Fig. 7, in the case of a scene in video content where fireworks are launched, the first to fourth oscillators 21 to 24 are normally used, and the realistic vibration level of each oscillator is adjusted to provide vibration to the user so that vibration occurs in the direction of the fireworks (for example, the first oscillator 21 and second oscillator 22 are strong vibrations, and the third oscillator 23 and fourth oscillator 24 are weak vibrations, and the vibration localization point is set closer to the front (the direction of the fireworks in the content)). Then, when notifying information as shown in Fig. 4, the realistic vibration is output using only the first oscillator 21 and third oscillator 23, which are not used for notification vibration. In other words, multiple vibrators suitable for information notification (second vibrator 22, fourth vibrator 24) are used preferentially to provide directional information to the notification vibration, and vibrators not used for the notification vibration (first vibrator 21, third vibrator 23) are used to generate realistic vibration.

[0060] Although the sense of direction of the realistic vibration may be reduced due to limitations on the vibrators used, in this example, the notification vibration is prioritized because notification of information is often important. Alternatively, the vibrator used for notification vibration can also be used for realistic vibration with the realistic vibration level reduced.

[0061] In such a case, the number of vibrators used is reduced from four to two, and the vibration level is corrected based on the change in the vibrators used. For example, the realistic vibration for the second vibrator 22 is added to the realistic vibration for the first vibrator 21 (the vibrator adjacent to the second vibrator 22 is selected), and the realistic vibration for the fourth vibrator 24 is added to the realistic vibration for the third vibrator 23 (the vibrator adjacent to the fourth vibrator 24 is selected). This makes it possible to suppress fluctuations in the overall sense of presence and sense of position of the realistic vibration when a notification vibration is generated. Another suitable application example is a method of increasing the vibration level of each vibrator according to the ratio of vibrators used to output realistic vibrations to those not used, specifically, setting the vibration level as (total number of vibrators / number of vibrators used).

[0062] Subsequently, as shown in FIG. 5, when the distance from the host vehicle 6 to the other vehicle 61 on the right front increases and the direction of the other vehicle 61 becomes the same as the direction from the seating center P1 to the second oscillator 22, the controller 2 uses the second oscillator 22 for information notification and does not use the first oscillator 21, the third oscillator 23, or the fourth oscillator 24. In other words, the controller 2 increases the vibration amount of the second oscillator 22 and stops the vibration of the fourth oscillator 24, which had previously had a weak vibration amount. As a result, in response to the change in the direction of the other vehicle 61 relative to the host vehicle 6, the vibration localization point P2 moves forward from the position shown in FIG. 4 to the position shown in FIG. 5. In other words, the direction from the host vehicle 6 to the other vehicle 61 and the direction from the seating center P1 to the vibration localization point P2 become the same, allowing the user to recognize the direction of the other vehicle 61 relative to the host vehicle 6.

[0063] Thereafter, as shown in FIG. 6, when the other vehicle 61 changes course to the left and approaches the direction of travel (forward) of the host vehicle 6, the controller 2 uses the first oscillator 21 and the second oscillator 22 for information notification, but does not use the third oscillator 23 and the fourth oscillator 24. The controller 2 then increases the vibration amount of the second oscillator 22, stops the vibration of the fourth oscillator 24, which had previously been vibrating at a low vibration amount, and decreases the vibration amount of the first oscillator, which had not been vibrating until then. As a result, in response to the change in the direction of the other vehicle 61 relative to the host vehicle 6, the vibration localization point P2 moves leftward from the position shown in FIG. 5 to the position shown in FIG. 6. In this way, the user can recognize the situation of the other vehicle 61 approaching the host vehicle 6 from the right front in the direction of travel of the host vehicle 6.

[0064] In this way, when generating realistic vibrations and notification vibrations simultaneously, the controller 2 performs control to generate notification vibrations with priority (for example, by determining which vibrator to use with priority), so that the user can be sure to be aware of notification-based information notifications (such as the vehicle's surroundings) that require high awareness. Also, when generating realistic vibrations and notification vibrations simultaneously, different vibrators are used, so the user can easily distinguish whether the vibration is a realistic vibration related to the video or a notification vibration for information notification.

[0065] [4. Vibration fade in and out] Furthermore, when notifying information by vibration during a period in which the realistic vibration is being generated, the controller 2 fades in and fades out the realistic vibration.

[0066] FIG. 8 is an explanatory diagram of fade-in and fade-out states of the realistic vibration according to the embodiment. As shown in FIG. 8, when the periphery monitoring device 11 detects an object that is a candidate for vibration notification at time t1 during the period in which the realistic vibration is being generated (when the danger level of the object is equal to or greater than a first threshold (a danger level at which a dangerous state notification should be prepared) or a second threshold (a danger level greater than the first threshold and at which a dangerous state notification should be made)), the controller 2 gradually reduces and fades out the level of the realistic vibration. When the level of the realistic vibration reaches a predetermined threshold (a level at which the influence on the notification vibration can be compromised (set by a design developer, etc., based on experiments, etc.), referred to as a vibration notification standby level) at time t2, the controller 2 stops reducing the level of the realistic vibration and maintains that level. Note that when the object that is a candidate for vibration notification disappears (when the danger level of the object becomes less than the first threshold), the controller 2 gradually increases the level of the realistic vibration to the original state (the state at time t1) and fades it in. In addition, since the instantaneous vibration level of the realistic vibration varies depending on the playback status (playback position) of the content (the audio signal of the content), the level of the realistic vibration here is used to mean the maximum level that the realistic vibration can reach, which varies depending on the content.

[0067] In addition, calculation of the danger level of an object, detection and disappearance of objects that are candidates for notification, and determination of objects that should be notified are performed by the surrounding monitoring device 11, and the information processing device 1 (controller 2) acquires the data. Alternatively, a method in which the information processing device 1 acquires various data such as position data of other vehicles from the surrounding monitoring device 11 and performs calculations based on the data acquired by the information processing device 1 can be applied.

[0068] Next, when the notification vibration generation state is entered at time t3 (when the danger level of the object reaches the second threshold), the controller 2 reduces the level of the realistic vibration to a predetermined level in the notification vibration generation state (a level at which the impact on the notification vibration is negligible (for example, level 0: realistic vibration stopped), which is set by designers and developers based on experiments, etc.). Then, the controller 2 generates the notification vibration. Then, at t4 after the notification vibration is generated (after the notification ends), the controller 2 begins a process (fade-in) of gradually increasing the level of the realistic vibration to the original state (the state at time t1).

[0069] This fade-in and fade-out control prevents sudden changes in the level of the realistic vibration, preventing the user from feeling uncomfortable when receiving information notification. Furthermore, when notification vibration occurs, the impact of the realistic vibration level is reduced to a negligible level, allowing the user to reliably recognize the notification vibration. Furthermore, if the vibration is continuously output at a level that has virtually no impact when receiving information notification, the loss of realism due to the disappearance of the realistic vibration can be prevented.

[0070] [5. Processing performed by the controller] Next, a process executed by the controller 2 according to the embodiment will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of a process executed by the controller 2 according to the embodiment. The controller 2 outputs a realistic vibration signal to each vibrator to give the user realistic vibration while playing music or video in content, and the process in Fig. 9 is repeatedly executed in this state at intervals that result in appropriate operation.

[0071] In step S101, the controller 2 calculates the level of danger based on data indicating the situation around the vehicle from the surroundings monitoring device 11, etc., and determines whether the level of danger is equal to or greater than a first threshold (the level of danger at which notification of a dangerous state should be prepared).If the level of danger is less than the first threshold, the processing ends, and if the level of danger is equal to or greater than the first threshold, the controller 2 proceeds to step S102.

[0072] In step S102, the controller 2 starts fading out the realistic vibration, and proceeds to step S103. Through the process of step S102, the vibration level of the realistic vibration gradually decreases to the vibration notification standby level.

[0073] In step S103, the controller 2 starts a process of increasing the sound reproduction level of a specific frequency band of the acoustic sound (a frequency band corresponding to the realistic vibration signal) in order to compensate for the decrease in the realistic feeling caused by the fade-out of the realistic vibration, and then proceeds to step S104. By the process of step S103, the acoustic sound level of the specific frequency band of the acoustic sound gradually increases as the realistic vibration level decreases.

[0074] In step S104, the controller 2 calculates the risk level based on data indicating the situation around the vehicle from the periphery monitoring device 11, and determines whether the risk level is equal to or greater than a first threshold.If the risk level is less than the first threshold, the controller 2 proceeds to step S105, and if the risk level is equal to or greater than the first threshold, the controller 2 proceeds to step S107.

[0075] In step S105, the controller 2 starts fading in the realistic vibration, and proceeds to step S106. By the processing in step S105, the vibration level of the realistic vibration gradually increases to the level before the fade-out processing in S102.

[0076] In step S106, the controller 2 starts a process of reducing the sound reproduction level of a specific frequency band of the acoustic sound (a frequency band corresponding to the realistic vibration signal) in order to compensate for the increase in the realistic feeling due to the fade-in of the realistic vibration (the realistic feeling is reduced compared to before the fade-out started), and then ends the process. By the process of step S106, the acoustic sound level of the specific frequency band of the acoustic sound gradually decreases as the realistic vibration level increases.

[0077] In step S107, the controller 2 calculates the degree of danger based on data indicating the situation around the vehicle from the perimeter monitoring device 11, etc., and determines whether the degree of danger is equal to or greater than a second threshold (a degree of danger at which a dangerous state should be notified: greater than the first threshold).If the degree of danger is equal to or greater than the first threshold, the controller 2 proceeds to step S108, and if the degree of danger is less than the second threshold, the controller 2 returns to step S104.

[0078] In step S108, the controller 2 performs a process of reducing the realistic vibration (reducing the realistic vibration to a level where the effect on the notification vibration is negligible (for example, level 0: realistic vibration stopped)), The process proceeds to step S109. By the process of step S102, the vibration level of the realistic vibration is reduced to a level where the effect on the notification vibration can be ignored (for example, the realistic vibration is stopped).

[0079] In step S109, the controller 2 increases the sound reproduction level of a specific frequency band in the acoustic sound (a frequency band corresponding to the realistic vibration signal) to compensate for the decrease in the realistic feeling due to the reduction in the realistic vibration, and then proceeds to step S110. By the process of step S103, the acoustic sound level of the specific frequency band in the acoustic sound is reduced in accordance with the reduction in the realistic vibration level.

[0080] In step S110, the controller 2 selects a vibrator to be used for notification vibration based on the direction of the object of the risk level, and also selects a vibrator not to be used for notification vibration as a vibrator to be used for realistic vibration, and proceeds to step S111. That is, the vibrator to be used for notification vibration is preferentially selected.

[0081] In step S111, the controller 2 generates a notification vibration signal for each vibrator based on the direction of the object of danger level, etc., and proceeds to step S112. Then, in step S112, the controller 2 performs notification processing, specifically outputting a display signal, an audio signal, and a notification vibration signal that notify, for example, of the occurrence of danger (the occurrence of an object of high danger level around the vehicle) to the display device 13, the sound output device 14, and each of the vibrators 21 to 24, and proceeds to step S113. Note that once the information notification is completed (when a predetermined notification period (predetermined time length) ends or the danger state disappears), the output of the display signal, audio signal, and notification vibration signal stops.

[0082] In step S113, the controller 2 starts fading in the realistic vibration, and proceeds to step S114. By the processing in step S113, the vibration level of the realistic vibration gradually increases to the level before the fade-out processing in S102.

[0083] In step S114, the controller 2 starts a process of reducing the sound reproduction level of a specific frequency band of the acoustic sound (a frequency band corresponding to the realistic vibration signal) in order to compensate for the increase in the realistic feeling due to the fade-in of the realistic vibration (the realistic feeling is reduced compared to before the fade-out started), and then ends the process. By the process of step S114, the acoustic sound level of the specific frequency band of the acoustic sound gradually decreases to the level before the fade-out process of S102 as the realistic vibration level increases.

[0084] In the above process, the sound of the content is not faded in, faded out, or the sound level is not reduced when the information is notified, but it may be faded in, faded out, or the sound level may be reduced in the same way as with the realistic vibration. In this case, the user's awareness of the sound of the information notification will be improved.

[0085] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

[0086] [6. Notes] As an appendix, the features of the present invention are as follows. (1) An information processing device that outputs audio and realistic vibrations based on content, Detects vibration notifications, When notifying information by vibration, the volume level of a specific frequency band in the audio of the content is increased. Information processing device. (2) When notifying information by vibration, the realistic vibration is reduced. The information processing device according to (1) above. (3) Before notifying information by vibration, the realistic vibration is faded out; After notifying the information by vibration, the realistic vibration is faded in. The information processing device according to (1) or (2). (4) The acoustic sound is faded out before the information is notified by vibration. After the notification of information by vibration, the acoustic sound is faded in. The information processing device according to (1) or (2). (5) The acoustic sound is faded out before the information is notified by vibration. After the notification of information by vibration, the acoustic sound is faded in. The information processing device according to (3) above. (6) The realistic vibration is generated by the low frequency components of the acoustic sound. The information processing device according to any one of (1) to (5). (7) The specific frequency band is The frequency range is from 50Hz to 150Hz. The information processing device according to any one of (1) to (6). (8) An information processing method for outputting audio and realistic vibration based on content, comprising: Detects vibration notifications, When notifying information by vibration, the volume level of a specific frequency band in the audio of the content is increased. A computer-implemented information processing method. (9) An information processing program that outputs audio and realistic vibrations based on content, A process of detecting notification of information by vibration; When notifying information by vibration, a process of increasing the volume level of a specific frequency band in the acoustic sound of the content is performed. An information processing program executed by a computer. (10) An information processing device that outputs an acoustic sound and a realistic vibration based on content and notifies information by a notification vibration, Detecting the notification of information by the notification vibration; When the information is notified by the notification vibration, the volume level of a specific frequency band in the audio sound of the content is increased. Information processing device. (11) It is equipped with multiple vibrators that generate vibrations, The plurality of vibrators are used preferentially for generating the notification vibration. The information processing device according to (10) above. (12) using the plurality of vibrators to provide directional information to the notification vibration; The vibrator not used for the notification vibration is used to generate the realistic vibration. The information processing device according to (10) or (11). (13) The frequency band of the realistic vibration is a band from 30 Hz to 200 Hz, The frequency band of the vibration used to notify the information is between 70 Hz and 150 Hz. The information processing device according to any one of (10) to (12). [Explanation of symbols]

[0087] 1. Information processing equipment 2 Controller 3. Control Unit 4 Storage section 5 seats 6 Vehicle 11 Periphery monitoring device 11a Detection processing unit 12 Operating device 13 Display device 14 Sound output device 21 First oscillator 22 Second oscillator 23 Third oscillator 24 4th oscillator 31 Vibration control unit 32 Display control unit 33 Audio control section 41 Music Video Information 42 Information Processing Program 51 Seat area 52 Backrest 61 Other vehicles Area A P1 center of gravity P2 Vibration localization point

Claims

1. An information processing device that outputs audio and realistic vibrations based on content, Detects vibration notifications, When notifying information by vibration, the volume level of a specific frequency band in the audio of the content is increased. Information processing device.

2. When notifying information by vibration, the realistic vibration is reduced. The information processing device according to claim 1 .

3. Before notifying information by vibration, the realistic vibration is faded out; After notifying the information by vibration, the realistic vibration is faded in.

3. The information processing device according to claim 1.

4. The acoustic sound is faded out before the information is notified by vibration. After the notification of information by vibration, the acoustic sound is faded in.

3. The information processing device according to claim 1.

5. The acoustic sound is faded out before the information is notified by vibration. After the notification of information by vibration, the acoustic sound is faded in. The information processing device according to claim 3 .

6. The realistic vibration is generated by the low frequency components of the acoustic sound. The information processing device according to claim 1 .

7. The specific frequency band is The frequency range is from 50Hz to 150Hz. The information processing device according to claim 1 .

8. An information processing method for outputting audio and realistic vibration based on content, comprising: Detects vibration notifications, When notifying information by vibration, the volume level of a specific frequency band in the audio of the content is increased. A computer-implemented information processing method.

9. An information processing program that outputs audio and realistic vibrations based on content, A process of detecting notification of information by vibration; When notifying information by vibration, a process of increasing the volume level of a specific frequency band in the acoustic sound of the content is performed. An information processing program executed by a computer.

10. An information processing device that outputs an acoustic sound and a realistic vibration based on content and notifies information by a notification vibration, Detecting the notification of information by the notification vibration; When the information is notified by the notification vibration, the volume level of a specific frequency band in the audio sound of the content is increased. Information processing device.

11. It is equipped with multiple vibrators that generate vibrations, The plurality of vibrators are used preferentially for generating the notification vibration. The information processing device according to claim 10.

12. using the plurality of vibrators to provide directional information to the notification vibration; The vibrator not used for the notification vibration is used to generate the realistic vibration. The information processing device according to claim 10.

13. The frequency band of the realistic vibration is a band from 30 Hz to 200 Hz, The frequency band of the vibration used to notify the information is a band from 70 Hz to 150 Hz. The information processing device according to claim 10.

Images