Multisensory stimulation presenting system, and information processing apparatus

Abstract

To provide a multisensory stimulation presenting system and an information processing apparatus which improve a degree of experience obtained by presentation of sensual stimulation which is not based on presentation of any visual content and not sequential.SOLUTION: A multisensory stimulation presenting system 100 according to the present invention has a visual perception device 2 for presenting only visual stimulation, an auditory perception device 3 for presenting only auditory stimulation, a haptic device 4 for presenting only haptic stimulation, and an olfactory perception device 5 for presenting only olfactory stimulation. An information processing apparatus 10 according to the present invention accepts designation of a scene of content, and requests the visual perception device 2, the auditory perception device 3, the haptic device 4, and the olfactory perception device 5 to present sense stimulation associated with the scene. Specifically, the visual perception device 2 presents visual stimulation corresponding to the scene, the auditory perception device 3 presents auditory stimulation corresponding to the scene, the haptic device 4 presents haptic stimulation corresponding to the scene, and the olfactory perception device 5 presents olfactory stimulation corresponding to the scene.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a multisensory stimulation presentation system and an information processing device. [Background technology]

[0002] A technology that presents tactile and olfactory stimuli is known as a means for enhancing the sense of realism and impact when enjoying content such as videos. A device that presents such multiple sensory stimuli will be described as a multisensory stimuli presentation system. A multisensory stimuli presentation system presents tactile and olfactory stimuli in synchronization with the progress of the content.

[0003] There is known a technology that presents sensory stimuli in response to some active action, rather than a user receiving sensory stimuli by passively viewing content (see, for example, Patent Document 1). Patent Document 1 discloses a technology that presents tactile stimuli when the user moves into a preset area in a virtual reality (VR) space. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-36711 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional technologies, there is a problem that the range of experiences obtained by presenting sensory stimuli is narrowed. For example, if sensory stimuli are presented in synchronization with the progress of content that transitions in one direction on the time axis, such as conventional video content, the user is limited to an experience of passively obtaining sensory stimuli while watching all of the content in order. Furthermore, in content in which sensory stimuli are presented in association with position information in virtual space or real space, the user is limited to an experience of obtaining sensory stimuli when moving to a specified position.

[0006] In view of the above-mentioned problems, the present disclosure provides a technology that does not require the presentation of video content and improves the breadth of experiences that can be obtained by presenting non-sequential sensory stimuli. [Means for solving the problem]

[0007] The present disclosure relates to a multisensory stimulus presentation system that presents sensory stimuli corresponding to content to a user, the system having an information processing device, a visual device that presents only visual stimuli, an auditory device that presents only auditory stimuli, a tactile device that presents only tactile stimuli, and an olfactory device that presents only olfactory stimuli, wherein the information processing device accepts designation of a scene of the content, and requests the visual device, the auditory device, the tactile device, and the olfactory device to present sensory stimuli corresponding to the scene, the visual device presents the visual stimuli corresponding to the scene, the auditory device presents the auditory stimuli corresponding to the scene, the tactile device presents the tactile stimuli corresponding to the scene, and the olfactory device presents the olfactory stimuli corresponding to the scene. Effect of the Invention

[0008] It is possible to provide a technology that does not require the presentation of video content and that improves the breadth of experiences that can be obtained by presenting non-sequential sensory stimuli. [Brief description of the drawings]

[0009] [Figure 1] FIG. 13 is a diagram showing an example of an explanatory text for a puzzle-solving content. [Diagram 2] 1 is a schematic overall diagram of an example of a system configuration of a multisensory stimulation presentation system. [Diagram 3] 1 is a diagram for explaining an outline of the operation of the multisensory stimulation presentation system. [Figure 4] FIG. 1 is an external view of an example of a scene operation device. [Diagram 5] FIG. 1 is an external view of an example of a visual device. [Figure 6]FIG. 1 is an external view of an example of a hearing device. [Figure 7] FIG. 1 is an external view of an example of a haptic device. [Figure 8] FIG. 1 is an external view of an example of an olfactory device. [Figure 9] FIG. 2 is a diagram illustrating a hardware configuration of an example of a scene control device. [Figure 10] FIG. 2 is a hardware configuration diagram of an example of a scene operation device and a control unit for each sensory device. [Figure 11] FIG. 2 is a hardware configuration diagram of an example of a stimulation presentation unit of each sensory device. [Figure 12] 2 is a functional configuration diagram of an example showing functions of a scene operation device, a scene control device, and a control section of each sensory device. FIG. [Figure 13] 11 is a diagram illustrating a project file stored in a presentation stimulus information management data storage unit. FIG. [Figure 14] FIG. 11 is a diagram showing an example of format data stored in a format data storage unit; [Figure 15] FIG. 13 is a diagram showing an example of a setting screen for setting presentation stimulus information in association with a timeline. [Figure 16] FIG. 13 illustrates an example pull-down menu that accepts the selection of a cartridge number and scent intensity. [Figure 17] FIG. 13 illustrates an example pull-down menu that accepts selection of an audio file, a vibration intensity level, a temperature polarity, and a temperature level. [Figure 18] 11 is a sequence diagram of an example showing a process in which a scene control device requests each sensory device to present a sensory stimulus according to a scene number specified by a user. FIG. [Figure 19] FIG. 13 is a sequence diagram illustrating an example of a process in which each user of a sensory device arbitrarily presents and acquires a sensory stimulus again. [Figure 20] 11 is a sequence diagram of an example showing a process in which a scene control device requests each sensory device to present a sensory stimulus according to a scene number specified by a user. FIG. [Figure 21]1 is a schematic overall diagram of an example of a system configuration of a multisensory stimulation presentation system. [Figure 22] 2 is a functional configuration diagram of an example showing functions of a scene operation device, a scene control device, and a control section of each sensory device. FIG. [Figure 23] 11 is a sequence diagram showing an example of a process in which a scene control device requests each integrated sensory device to present a sensory stimulus according to a scene number specified by a user. FIG. [Figure 24] 11 is a sequence diagram illustrating an example of a process in which each user of the integrated sensory device obtains an arbitrary sensory stimulus in an arbitrary scene. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, as an example of an embodiment of the present disclosure, a multisensory stimulation presentation system and a method of presenting a sensory stimulation performed by the multisensory stimulation presentation system will be described with reference to the drawings.

[0011] <Additional information regarding presentation of sensory stimuli> Technologies that present tactile and olfactory stimuli are known as a means of enhancing the sense of realism and impact when users enjoy video content such as movies. Conventional multisensory stimulation presentation systems present tactile and olfactory stimuli in synchronization with the progression of video content.

[0012] In addition to content provided with a set progression, there are many experiential contents that involve active actions, such as mystery nights, real escape games, and immersive theaters. These contents require users to move between locations and specific situations (scenes), or to go back to past events to search for clues. There are also contents in which the story experienced changes depending on the user's actions. With current technology, realizing such experiences requires extensive manual preparation and human movement. Even for content such as games that involve moving around in a VR space, users need to perform many operations and time to move to specific locations and trigger events, and performing such tasks for long periods of time while wearing a head-mounted display places a strain on the user.

[0013] Furthermore, a method that always assumes the presentation of video content and relies on a multisensory stimulation presentation system to present sensory stimuli such as touch and smell synchronized with the video content means that only able-bodied people with no impairments in any of their senses can enjoy it.

[0014] Therefore, in this embodiment, information on presentation stimuli (hereinafter referred to as presentation stimulus information) designated for vision, hearing, smell, touch, etc. is prepared for each scene of the content, and when the user transitions between scenes (scenes do not need to be transitioned in chronological order), the multisensory stimulus presentation system presents the sensory stimuli associated with the scene, thereby realizing an active multisensory experience. The presentation stimulus information is made up of data such as visual stimulus material files (video files, image files, etc.) and auditory stimulus material files (audio files, etc.) that are components, vibration patterns related to tactile stimuli, vibration intensity levels, temperature polarity and temperature levels, and the type of scent and signal intensity level of olfactory stimuli. When the user specifies a scene to be presented, a control signal corresponding to the presentation stimulus information associated with the scene is transmitted to the sensory device for presentation, and the sensory device presents the designated sensory stimulus. In addition, each sensory device can present the sensory stimulus associated with the scene again, using a user operation as a playback trigger. The presentation stimulus information is rewritable, and the administrator can easily edit the stimulus information presented in each scene.

[0015] In this way, this embodiment allows the user to call up each scene and present sensory stimuli with a simple operation, making it possible for the user to actively transition between scenes and present sensory stimuli. In addition, the administrator can edit, store, and present each presentation stimulus information for each scene of the content.

[0016] Next, one of the usage scenes of the multisensory stimulation presentation system described in this embodiment will be described. In many attempts to improve the realism and impact of content, tactile and olfactory stimulations are additionally presented in conjunction with video content. However, in the production of content using multiple sensory stimulations, it is possible to create new experiences such as presenting only one or more arbitrary senses, in addition to the method of presenting all of them simultaneously in conjunction with each other.

[0017] In this embodiment, we propose to use content with puzzle-solving elements that can be played by multiple people, and to encourage communication between users by limiting the senses through which information can be obtained to one per person, thereby realizing an active experience. Users with impairments in any of the senses can also participate.

[0018] Each user is responsible for one of the senses - sight, hearing, touch, or smell - and obtains sensory stimuli using their own sensory device. Each sensory device is controlled by a scene control device (described later) to switch content scenes. Users obtain information from the sense they are responsible for in each scene of the content, and can solve the mystery by communicating and sharing the information they have obtained.

[0019] Figure 1 is an explanatory text for an example of puzzle-solving content. This puzzle-solving content is content in which the target "animal" is derived based on information obtained from the presented sensory stimuli. The correct answer is a hippopotamus. Details of the system will be described later, but scenes within the content are switched by a scene operation device, which will be described later. When a user performs a scene switching operation on the scene operation device, information about the scene (scene number) is sent to the scene control device. Each sensory device presents a sensory stimuli based on a control signal notified by the scene control device. If a request is made from each sensory device (by pressing a button) after the scene is switched, the sensation set for the scene after the switch will be presented again.

[0020] For example, in the puzzle-solving content shown in Figure 1, the scene operation device is likened to a "clock that can operate the time." When the user changes the time on the clock, the information obtained by request (pressing a button) from each sensory device changes depending on the animal's habits (such as nocturnal), and it becomes a hint to think about the target animal. One of the participants (for example, the person in charge of vision) is in charge of operating the scene operation device, and the participants decide the time to switch by consulting with each other. When the time is switched, a speaker connected to the scene control device (or scene control device) announces "The time has been changed." For example, hippos are nocturnal animals that live in Africa, and are said to spend the day in the water and come up on land at night to eat grass. Therefore, if the correct answer is a hippopotamus, during the daytime, visual information is presented as "pitch black" (because it is sleeping), auditory information is presented as "sound of water," tactile information is presented as "vibration of water and cold temperature," and olfactory information is presented as "no odor." If the visual engineer sets it to nighttime, the visual information will be "a dark image of a waterside with grass," the auditory information will be "the sound of grass being eaten," the tactile information will be "the vibration of grass being eaten and a slightly warm temperature," and the olfactory information will be "the smell of grass."

[0021] <Terminology> Sensation refers to the ability to sense stimuli from the outside world (such as the five senses) and the phenomenon of consciousness that occurs when the external stimuli captured by these senses reach the brain center. In this case, when each sensory device controls the presentation of sensory stimuli to the user, it will be expressed as "presenting a sensory stimulus" or "presenting a stimulus," and the information on presented stimuli used in content management will be expressed as "presented stimulus information."

[0022] The presented stimulus information is information that specifies what kind of sensory stimulus is to be presented. The presented stimulus information is preferably prepared in a format that can be edited by an administrator, such as a CSV file described later.

[0023] The content is information content that is transmitted through media such as electronic devices, networks, paper, etc. In this embodiment, the content refers to information that has a presentation stimulus or is the presentation stimulus itself and is used to present a sensory stimulus.

[0024] A scene refers to a section (a certain range) of content. A scene may also be called a plot. A scene does not necessarily have to follow a story division. For example, a scene may be a division of content at a certain time interval (for example, time).

[0025] A visual device is a device that presents only visual stimuli, an auditory device is a device that presents only auditory stimuli, a tactile device is a device that presents only tactile stimuli, and an olfactory device is a device that presents only olfactory stimuli, although each device may have a function to present other senses or multiple sensory stimuli.

[0026] [First embodiment] <System configuration example> Next, the system configuration of the multisensory stimulus presentation system 100 will be described with reference to Fig. 2 and Fig. 3. Fig. 2 shows a schematic overall view of the system configuration of the multisensory stimulus presentation system 100, and Fig. 3 is a diagram for explaining an outline of the operation of the multisensory stimulus presentation system 100.

[0027] 2, a scene control device 10, a plurality of sensory devices 6, and a scene operation device 1 are communicatively connected via a network N. The network N may be, for example, a wireless LAN or a wired LAN, but may also include the Internet. Communication using Bluetooth (registered trademark) may also be used.

[0028] The scene control device 10 is an information processing device that controls the entire multisensory stimulation presentation system 100. The scene control device 10 accepts the start, end, etc. of the sensory stimulation presentation process. In response to a scene switching request from the scene operation device 1, the scene control device 10 creates a control signal for presenting a sensory stimulation and transmits it to each sensory device 6. The scene control device 10 can permit or restrict the scene operation device 1 and the sensory device 6 to select a scene and present a sensory stimulation. The scene control device 10 may be a general PC (Personal Computer).

[0029] The sensory devices 6 are a visual device 2, an auditory device 3, a tactile device 4, and an olfactory device 5. When there is no need to distinguish between the scene operation device 1, the visual device 2, the auditory device 3, the tactile device 4, and the olfactory device 5, they are simply referred to as "devices." When there is no need to distinguish between the visual device 2, the auditory device 3, the tactile device 4, and the olfactory device 5, they are simply referred to as "sensory devices 6." All of the devices may be able to communicate with the scene control device 10. Furthermore, communication may be wired or wireless, but the following description will be given assuming wireless communication.

[0030] The scene operation device 1 accepts a scene switching operation in response to a user's operation, and requests the scene control device 10 to switch the scene. When a scene switching operation is performed on the scene operation device 1, a scene number (scene identification information) indicating the corresponding scene is transmitted to the scene control device 10. The scene control device 10 holds a project file for each content. The project file is setting information that associates a scene with information on a sensory stimulus to be presented by each sensory device 6 (see FIG. 13).

[0031] Each sensory device 6 has a control unit 201 and a stimulus presentation unit 202. The control unit 201 is a control unit for the stimulus presentation unit 202, and the hardware configuration may be common to the scene operation device 1 as well. The stimulus presentation unit 202 is a presentation device for presenting each sensory stimulus, and has a configuration suitable for the sensory stimulus to be presented. In FIG. 2, the control unit 201 and the stimulus presentation unit 202 are separate entities, but the two may be integrated. Details of the control unit 201 and the stimulus presentation unit 202 will be described later.

[0032] 2, the scene operation device 1 and the scene control device 10 are provided as separate devices, but they may be one device. In this case, the scene control device 10 directly accepts scene selection from the user. If the scene control device 10 is a PC, the scene may be selected by input from the UI on the PC screen, etc.

[0033] In addition, the visual device 2, the scene operation device 1, and the scene control device 10 may be integrated. The auditory device 3, the scene operation device 1, and the scene control device 10 may be integrated. The tactile device 4, the scene operation device 1, and the scene control device 10 may be integrated. The olfactory device 5, the scene operation device 1, and the scene control device 10 may be integrated. In other words, each sensory device 6 presents only one sensory stimulus, but may have the functions of the scene control device 10 and the scene operation device 1. Each sensory device 6 accepts the selection of a scene, creates a control signal for presenting a sensory stimulus according to the scene number, and controls the stimulus presentation unit 202.

[0034] The following description will be based on FIG. (1) The user operates the scene operation device 1. The scene operation device 1 transmits a scene number to the scene control device 10. (2) The scene control device 10 refers to the project file and identifies the presentation stimulus information associated with the scene number. As shown in Fig. 13, the presentation stimulus information is the file name of a video or sound in the case of vision and hearing, the name of a file in which an audio signal used for vibration presentation is recorded, vibration intensity level, temperature polarity and temperature level in the case of touch, and the number of a cartridge (containing essential oils, etc.) that is the source of a scent and scent intensity in the case of smell. The information specified here differs depending on the implementation style of each sensory device. It is preferable that the scene control device 10 has a speaker and a display and announces that the scene has changed. (3) The scene control device 10 transmits a control signal to each sensory device 6 to present a sensory stimulus corresponding to the presentation stimulus information so that the sensory stimulus set in the scene is presented. In other words, the control signal is a signal to present the sensory stimulus of (2). Each sensory device 6 has a material file (visual: image file, etc., auditory: audio file, etc., tactile: file in which audio signals used for vibration presentation are recorded, etc.) designated in advance by a file name on a recording medium or the like, and also has a scent element (smell) in a cartridge format or the like. Note that, except for smell, each material file may be in a format that is not in advance held by each sensory device 6, but may be in a format that is transmitted from the scene control device 10 in response to a scene call. The sensory device 6 controls the sensory stimuli to be presented in response to the control signal. In addition, each sensory device 6 changes the screen or outputs a voice message depending on the state, such as "Scene updating -> Scene presenting".

[0035] <Appearance examples of each device> Next, specific examples of each device will be described with reference to Fig. 4 to Fig. 8. Fig. 4 is an external view of a scene operation device 1. The scene operation device 1 has a microcomputer module 11, a display 12, a button (with a rotary encoder) 13, a speaker 14, a power supply, etc. The display 12 displays a screen so that the user can see the current scene status, etc., and the speaker 14 plays announcements related to the scene.

[0036] The scene operation device 1 has a button 13 with a built-in rotary encoder so that the user can easily operate it. In this embodiment, the scene operation device 1 used for scene operation is described as having a rotary encoder, but other sensory devices 6 may also have a similar button 13. The button 13 can be rotated and pressed. When the user rotates the button 13 (rotary encoder), the scene changes according to the amount of rotation. The scene operation device 1 can support scene selection with information output by the display 12 and the speaker 14. For example, by using the button 13 with a rotary encoder, when the user rotates the button 13, the scene number (the number assigned to each scene) displayed on the display 12 changes in tandem, or the speaker announces "Scene 1", "Scene 2", etc. For example, in FIG. 4, the scene number displayed in the center of the display 12 changes according to the amount of rotation of the button 13. When the user presses the button 13, the scene with the scene number displayed in the center of the display 12 (the scene number last announced from the speaker) is determined, and the scene number is transmitted to the scene control device 10.

[0037] FIG. 5 shows an external view of the visual device 2, FIG. 6 shows an external view of the auditory device 3, FIG. 7 shows an external view of the tactile device 4, and FIG. 8 shows an external view of the olfactory device 5. Each sensory device 6 has a control unit 201 and a stimulus presentation unit 202 (respectively denoted by reference numerals 30, 36, 37, and 45) as a common configuration. The control unit 201 has a microcomputer module 21, a display 22, a button 23, a speaker 24, a power supply, and the like. When the control unit 201 is presenting a sensory stimulus or communicating with the scene control device 10, still images and videos that indicate the state are displayed on the display 22. The control unit 201 has a recording medium (a flash memory such as a microSD) that can be accessed from the microcomputer module 21. The recording medium stores material files (visual: image files, etc., auditory: audio files, etc., tactile: audio files in which audio signals used for vibration presentation are recorded, etc.). When the microcomputer module 21 receives a control signal from the scene control device 10, it controls the stimulus presentation unit 202 in accordance with the content of the control signal. The stimulus presentation unit 202 of each sensory device 6 will be described below.

[0038] The control unit 201 may be replaced by a smartphone, a tablet terminal, a PC, etc. In this case, an application that provides the functions of the control unit 201 is installed in the smartphone, tablet terminal, or PC.

[0039] The stimulus presentation unit 202 of the visual device 2 is a display device 30. The display device 30 and the control unit 201 are connected by an HDMI (registered trademark) cable or the like. The display device 30 has a speaker 31, a button 32, and a display 33. The display device 30 is assumed to be a mobile display with a touch function, a tablet computer, a microcomputer equipped with a display, or the like. The button 32 may be a physical button or a soft key displayed on the display 33. When the stimulus presentation unit 202 of the visual device 2 is a tablet computer, software for presenting visual stimuli (e.g., video playback software) is started on the tablet computer.

[0040] A visual stimulus presentation area 34 and buttons 32 (if a soft key is provided) are displayed on the display 33. When the control unit 201 receives a control signal (including information on the name of a material file) due to a scene change, it retrieves a material file corresponding to the changed scene from a storage medium and transmits it to the display device 30, so that the display device 30 can present a visual sensory stimulus (such as a still image or a video) in the visual stimulus presentation area 34. The display 33 may have a mounting section for a storage medium. When the user presses the button 32 (or the button 23), the sensory stimulus is presented again. This is because content that requires active behavior, such as puzzle-solving content, may need to be played back multiple times.

[0041] The stimulus presentation unit 202 of the auditory device 3 is, for example, an earphone 36. The control unit 201 of the auditory device 3 includes an audio module, an amplifier module, an earphone jack 35, etc., and has an audio output function. The earphone 36 is connected to the earphone jack 35. The earphone 36 may be an audio playback device including an amplifier module, etc. In addition, the example of FIG. 6 shows an example in which the control unit 201 has a recording medium, but the earphone 36 may have a mounting section for the recording medium. When the control unit 201 receives a control signal (including information on the material file name) due to scene switching, it obtains a material file corresponding to the switched scene and presents a sensory stimulus for the auditory sense (by playing back an audio file, etc.). In addition, similar to the visual device 2, when the user presses the button 23, the sensory stimulus is presented again.

[0042] The stimulus presentation unit 202 of the haptic device 4 is the haptic stimulus presentation device 37. The control unit 201 includes an audio module for outputting an audio signal used for vibration presentation, an amplifier module, etc., and the haptic stimulus presentation device 37 and the control unit 201 are connected by an electric wire / cable, etc. The control unit 201 also includes a recording medium. The haptic stimulus presentation device 37 includes a stimulus presentation plate 41 having a material with high thermal conductivity (such as graphite), a vibrator 42 that vibrates the stimulus presentation plate 41, a Peltier element 43 that can present a warm or cold tactile sensation, a driver (such as a motor driver) for controlling the Peltier element 43, and a power source. The vibrator 42 and the Peltier element 43 are attached to the back surface of the stimulus presentation plate 41. The vibrator 42 can vibrate at various frequencies, for example, and present various tactile sensations. The stimulus presentation unit 202 of the haptic device 4 may include a recording medium. In the example of FIG. 7, the recording medium is stored in the same housing as the control unit 201.

[0043] When the control unit 201 receives a control signal (including information on the material file name, vibration intensity level, temperature polarity, and temperature level) due to scene switching, the control unit 201 acquires information on vibration (information on the audio file in which the signal used for vibration presentation is recorded and information on the vibration intensity level) from the presentation stimulus information corresponding to the switched scene, plays the audio file at a volume that will result in a specified vibration intensity level, and transmits the signal to the tactile stimulus presentation device 37. The tactile stimulus presentation device 37 vibrates the vibrator 42 with the received signal. In addition, the control unit 201 transmits information on temperature (temperature polarity and temperature level) corresponding to the switched scene to the tactile stimulus presentation device 37. The tactile stimulus presentation device 37 controls the polarity of the current that controls the Peltier element 43 according to the specified temperature polarity, controls either hot or cold, and controls the Peltier element 43 by controlling the current so that the temperature level is the specified level. This presents tactile information corresponding to the scene. In addition, as with the other sensory devices 6, when the user presses the button 23, the tactile stimulus can be presented again. The tactile stimulation device 37 may have an audio signal output function (audio file playback function).

[0044] The stimulation presentation unit 202 of the olfactory device 5 has an olfactory stimulation presentation device 45 (such as Aroma Shooter (registered trademark) manufactured by Aroma Join Co., Ltd.) in which an aroma element is loaded. The olfactory stimulation presentation device 45 and the control unit 201 communicate with each other, for example, wirelessly. The olfactory stimulation presentation device 45 in FIG. 8 may communicate with the control unit 201 wirelessly or by wire. The olfactory stimulation presentation device 45 is provided with more aroma elements (in cartridge form in an example using an aroma shooter) than the types of aromas presented in each content. When the control unit 201 receives a control signal (including cartridge number and intensity information) due to a scene change, it controls the olfactory stimulation presentation device 45 with olfactory presentation stimulation information (such as the cartridge number and aroma intensity of the aroma) corresponding to the changed scene. The olfactory stimulation presentation device 45 presents the specified aroma at the specified intensity. As with the other sensory devices 6, the user can press the button 23 to present the aroma again.

[0045] <Hardware configuration example> Next, the hardware configuration of the scene control device 10 and the devices will be described.

[0046] <<Scene Control Device>> Fig. 9 is a diagram showing an example of a hardware configuration of the scene control device 10 according to this embodiment. As shown in Fig. 9, the scene control device 10 is constructed by a computer 500, and includes a CPU 501, a ROM 502, a RAM 503, a HD (Hard Disk) 504, a HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, an optical drive 514, a media I / F 516, and a speaker 517.

[0047] Among these, the CPU 501 controls the operation of the entire computer 500. The ROM 502 stores programs such as IPL used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data from the HD 504 under the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, a character, or an image. The external device connection I / F 508 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 509 is an interface for data communication using the network N. The bus line 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501 shown in FIG. 9.

[0048] Moreover, the keyboard 511 is a type of input means having a plurality of keys used for inputting characters, numbers, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The optical drive 514 controls reading and writing of various data from an optical storage medium 513, which is an example of a removable storage medium. The optical storage medium 513 is a CD, a DVD, Blu-ray (registered trademark), etc. The media I / F 516 controls reading and writing (storing) of data from a storage medium 515 such as a flash memory. The speaker 517 outputs sound.

[0049] <<Scene operation device and sensory device control section>> Next, the hardware configuration of the scene operation device 1 and the control unit 201 of each sensory device 6 will be described with reference to Fig. 10. Fig. 10 shows a hardware configuration diagram of the scene operation device 1 and the control unit 201 of each sensory device 6, mainly showing the computer part.

[0050] The control unit 201 has microcomputer modules 11, 21, displays 12, 22, speakers 14, 24, buttons 13, 23, and a storage medium mounting unit 25. The microcomputer modules 11, 21 are also called computers, and are devices in which an arithmetic and control device (CPU 211), memory devices (RAM 212 and ROM 213), a timer circuit 214, an input / output circuit (I / O 215), and the like are implemented in a single integrated circuit. It is preferable that the microcomputer modules 11, 21 have a wireless communication function such as wireless LAN or Bluetooth (registered trademark). However, the microcomputer modules 11, 21 may communicate with the scene control device 10 via a wired LAN or the like.

[0051] The microcomputer modules 11 and 21 are modules that have a full range of computer functions by themselves, and are often more compact than computers such as PCs that have independent components such as a CPU, RAM, ROM, etc. However, there is no problem if the control unit 201 has an independent CPU, RAM, ROM, etc.

[0052] The displays 12 and 22 are devices that light up pixels in response to video signals to display characters, figures, graphics, videos, etc., and are sometimes called monitors. The displays 12 and 22 are liquid crystal or organic electroluminescence displays, but may also be displays dedicated to displaying characters. The type and size of the displays 12 and 22 may differ depending on the sensory device 6.

[0053] The speakers 14 and 24 notify the user of a scene change by sound or present the current state. Instead of the speakers 14 and 24, or in addition to the speakers 14 and 24, an earphone jack may be mounted on the control unit 201. Also, sound may be transmitted by wireless communication such as Bluetooth (registered trademark).

[0054] The buttons 13 and 23 are, for example, alternate buttons that are pressed down by a user operation and return to their original state by a reaction force. The buttons 13 and 23 accept various operations on the control unit 201. A rotary encoder may be integrally mounted on the buttons 13 and 23. By rotating the rotary encoder, a list of options is displayed on the displays 12 and 22 while being switched.

[0055] The storage medium mounting unit 25 is an interface into which an SD card (registered trademark), a micro SD card, a USB memory, etc. can be detachably attached. A storage medium in which material files (for vision, hearing, and touch) are stored may be attached to the storage medium mounting unit 25. The storage medium used by the control unit 201 does not have to be of such a portable type, and a built-in memory device may also be used.

[0056] <<Sensory presentation section of each sensory device>> Fig. 11 shows a hardware configuration diagram of the stimulus presentation unit 202 of each sensory device 6. As described above, the stimulus presentation unit 202 differs depending on the sensory device 6, and Fig. 11 shows the main configuration of the sensory device 6. The stimulus presentation unit 202 is the display device 30, the earphones 36, the tactile stimulus presentation device 37, and the olfactory stimulus presentation device 45, but Fig. 11 mainly shows the tactile stimulus presentation device 37 and the olfactory stimulus presentation device 45.

[0057] The stimulus presentation unit 202 has a drive circuit 27 and various actuators 28 dedicated to the sensory device 6. The various actuators 26 may not be present in the visual device 2 and the auditory device 3. The drive circuit 27 is a circuit for driving the various actuators 28, and controls the actuators by, for example, the duty ratio and current direction of a PWM signal. For example, the vibration intensity level of the vibrator 42, the temperature polarity and temperature level of the Peltier element 43, etc. are controlled by the drive circuit 27.

[0058] In the case of the haptic device 4, the various actuators 28 are a vibrator 42 and a Peltier element 43. The vibrator 42 has, for example, a spring resonant structure inside, and generates vibration by passing electricity through a vibrating body of the spring resonant structure by electromagnetic induction to magnetize it. If the vibrator 42 has multiple resonance points and thus has two or more axes of vibration freedom, such as the HAPTIC (registered trademark) Reactor Hybrid Tough Type (Alps Alpine), a complex waveform sensation can be reproduced. The vibrator 42 may be an ERM (eccentric motor) or an LRA (linear resonant actuator).

[0059] A Peltier element is a plate-shaped semiconductor thermoelectric element that uses the Peltier effect. When a direct current is passed through a Peltier element in a certain direction, the upper surface of the element absorbs heat (cools) and the lower surface generates heat (heats). In addition, when the direction of the direct current is changed, the cooling and heating surfaces of the Peltier element are switched, and the element has the characteristic of quick response to the set temperature.

[0060] In the case of the olfactory device 5, the various actuators 28 are air compressors, air blowers, etc. that output the scent. The various actuators 28 control the type of scent by opening and closing the scent storage section and controlling the air flow path, and control the intensity of the scent by the amount of compression and the strength of the air blow.

[0061] For the visual device 2, a terminal such as an HDMI (registered trademark) is provided in the control unit 201, and content corresponding to the scene is displayed on a display 33 connected to the terminal. The content may be displayed on a display 22 possessed by the control unit 201. For the auditory device 3, an earphone jack 35 is provided in the control unit 201, and content corresponding to the scene is output from an earphone 36 connected to the earphone jack 35.

[0062] <About the function> Next, functions of the scene operation device 1, the scene control device 10, and each sensory device 6 will be described with reference to Fig. 12. Fig. 12 is a functional configuration diagram showing functions of the scene operation device 1, the scene control device 10, and the control unit 201 of each sensory device 6.

[0063] <<Scene operation device>> The scene operation device 1 has a communication unit 51, a display control unit 52, an operation reception unit 53, and a scene number storage unit 59. Each of these functional units is a function or means realized by developing a program stored in a ROM 213 of the microcomputer module 11 of the scene operation device 1 into a RAM 212 and causing a CPU 211 to execute each command of the program.

[0064] The communication unit 51 communicates with the scene control device 10 by wireless communication such as wireless LAN or Bluetooth (registered trademark) or wired communication such as wired LAN or USB cable. The communication unit 51 transmits to the scene control device 10 a scene number designated by the user.

[0065] The display control unit 52 controls information to be displayed on the display 12 of the scene operation device 1. The display control unit 52 displays on the display 12 a scene number (obtained from a scene number storage unit 59) corresponding to the amount of rotation of the button 13 (rotary encoder) accepted by the operation acceptance unit 53, and displays the currently selected scene number.

[0066] The operation reception unit 53 receives operations performed by the user on the scene operation device 1, such as operations on the buttons 13 including a rotary encoder.

[0067] The scene number storage unit 59 stores scene numbers of the content. For example, if the content is divided into scenes 1 to 10, the scene number storage unit 59 stores scene numbers such as scene 1, scene 2, ... scene 10. The scene numbers may be transmitted from the scene control device 10 at the start of the sensory stimulus presentation process, or may be stored in a storage medium in advance, and the storage medium may be attached to the storage medium attachment unit 25 of the scene operation device 1.

[0068] <<Scene Control Device>> The scene control device 10 has a communication unit 61, a display control unit 62, an operation reception unit 63, an audio output unit 64, a scene control unit 65, a presentation stimulus information management data storage unit 68, and a format data storage unit 69. Each of these functional units is a function or means realized when a program stored in the HD 504 of the scene control device 10 is expanded in the RAM 503 and the CPU 501 executes each command of the program.

[0069] The communication unit 61 communicates with the scene operation device 1 and each sensory device 6 via wireless communication such as wireless LAN or Bluetooth (registered trademark) or wired communication such as wired LAN or USB cable. For example, the communication unit 61 receives a scene number designated by the user from the scene operation device 1. In addition, the communication unit 61 transmits a control signal created based on presentation stimulus information associated with the scene to each sensory device 6.

[0070] The display control unit 62 controls the information to be displayed on the display 506 of the scene control device 10. The display control unit 62 displays a setting screen and the like relating to the operation of the multisensory stimulation presentation system 100. The display control unit 62 also displays the file name and contents of the project file, the currently selected scene number, and the like. The display control unit 62 also displays messages such as "Scene 2 has been reached" and "Currently, Scene 2 is displayed."

[0071] The operation reception unit 63 receives user operations on the scene control device 10. The scene control device 10 is mainly operated by an administrator. When the administrator wants to start providing an active multisensory experience to each user (sensory stimulus presentation processing), he or she inputs a start operation. Also, when the sensory stimulus presentation processing is set not to end automatically, the administrator performs an operation to end it. The sensory stimulus presentation processing may end automatically when the specified sensory stimuli have been presented in all scenes or when a set time has elapsed.

[0072] For example, when the scene is switched (when the audio output unit 64 receives the scene number from the scene operation device 1 and transmits a control signal to each sensory device 6), it outputs from the speaker 517 "Scene 2 has begun" or "You are currently in scene 2".

[0073] The scene control unit 65 reads the presentation stimulus information associated with the scene number transmitted from the scene operation device 1 from the presentation stimulus information management data storage unit 68. The scene control unit 65 transmits a control signal for presenting a sensory stimulus to the sensory device 6. The control signal may be data including as its contents a material file name in the case of vision and hearing, a material file name, vibration intensity level, temperature polarity and temperature level in the case of touch, and a scent cartridge number and scent intensity in the case of smell.

[0074] The scene control unit 65 interprets the presentation stimulus information in the presentation stimulus information management data storage unit 68 in reference to the format data storage unit 69. The presentation stimulus information management data storage unit 68 and the format data storage unit 69 will be described with reference to Figs. 13 and 14, respectively.

[0075] <<Project file>> FIG. 13 is a diagram for explaining a project file stored in the presentation stimulus information management data storage unit 68. FIG. 13(a) shows a state in which the scene control device 10 displays a selected project file on a UI screen. The screen of FIG. 13(a) is also an editing screen for the project file. As shown in FIG. 13(a), information (such as a material file and intensity) used to present a sensory stimulus associated with each of the senses of vision, hearing, touch, and smell is called presentation stimulus information. An mp4 file in the visual item is visual stimulus information, an mp3 file in the auditory item is auditory stimulus information, "mp3 file: warm / cold (temperature polarity): numerical value (vibration intensity level): numerical value (temperature level)" in the tactile item is tactile stimulus information, and "numerical value (cartridge number): numerical value (intensity)" in the olfactory item is olfactory stimulus information. The presentation stimulus information specifies what kind of sensory stimulus each sensory device 6 presents.

[0076] The scene control device 10 displays a data file that summarizes the presentation stimulus information for all scenes in the content as a "project file." The scene control device 10 controls the sensory stimuli to be presented for each scene by referring to the project file. As an example, as shown in FIG. 13(b), the project file is described in a CSV file. The project file in FIG. 13(b) is in CSV format, but may be in a table format, JSON format, XML format, or the like.

[0077] In the project file, presentation stimulus information to be presented in each scene is set for each sense. When the scene control device 10 receives a scene number from the scene operation device 1, it can identify the presentation stimulus information corresponding to the scene number.

[0078] In FIG. 13(a), visual, auditory, tactile, and olfactory presentation stimulus information is specified for each of five scenes with scene numbers 0 to 4. Presentation stimulus information for the sixth and subsequent scenes is displayed by scrolling to the right. For example, when the scene operation device 1 switches to scene 2, the visual device 2 plays and presents the video file "2.mp4". The auditory device 3 plays and presents the audio of "2.mp3". The tactile device 4 presents the signal of the audio file "2.mp3" and vibrations caused by the vibrator 42 vibrating at a vibration intensity level of 1. The tactile device 4 also presents a warm sensation with a temperature polarity (warm) of temperature level 4. The olfactory device 5 presents the scent of No. 2 (cartridge number) at an intensity of 50 (assuming that the intensity can be specified from 0 to 100 in the specifications of the olfactory device 5).

[0079] Furthermore, presentation of sensory stimuli corresponding to the scenes can also be performed by the scene control device 10. When the manager presses the play button 302 displayed in each scene, a control signal is transmitted to each sensory device 6.

[0080] The administrator can edit the visual, auditory, tactile, and olfactory presentation stimulus information for each scene. When sensory stimuli are presented by playing mp3 or mp4 files, the administrator refers to the file names of mp3 or mp4 files (e.g., including the scene number in the file name) prepared in advance for each scene and registers file names such as "scene number.mp3" and "scene number.mp4" in the project file. The main body of the material file is stored in advance in the storage medium of the sensory device 6. However, the scene control device 10 may transmit the material file itself together with the control signal to each sensory device 6. In addition, the administrator can directly edit the vibration intensity, temperature polarity and temperature level, and cartridge number and intensity. The options may be displayed in a pull-down menu as shown in FIG. 16, which will be described later.

[0081] 13(a) is a button that causes the scene control device 10 to start a sensory stimulus presentation process (provision of an active multisensory experience). When the communication start button 301 is pressed, the scene control device 10 waits for a scene number from the scene operation device 1.

[0082] 14 shows an example of the format data stored in the format data storage unit 69. The description format for each sensory stimulus is as follows. Visual: Video file to display (.mp4) Hearing: Audio file to play (.mp3) Tactile: Audio file (.mp3) containing the audio signal to be input to the transducer; Temperature polarity (warm, cold, none); Vibration intensity level (1-4); Temperature level (1-4) Olfactory: Cartridge number (1-6); Intensity (0-100) The scene control unit 65 interprets the presentation stimulus information in CSV format based on such format data, and transmits a control signal to each sensory device 6.

[0083] <<Control section for each sensory device>> Returning to Fig. 12, the control unit 201 of the sensory device 6 has a communication unit 71, a display control unit 72, an operation reception unit 73, a device control unit 74, and a device control program storage unit 79. Each of these functional units is a function or means realized when a program stored in a ROM 213 of the microcomputer module 21 of the sensory device 6 is expanded into a RAM 212 and each command of the program is executed by a CPU 211.

[0084] The communication unit 71 communicates with the scene control device 10 by wireless communication such as wireless LAN or Bluetooth (registered trademark) or by wired communication such as wired LAN or USB cable. The communication unit 71 receives a device control signal corresponding to a scene number designated by the user.

[0085] The display control unit 72 causes the display 22 to display information corresponding to the state of the sensory device 6. The display control unit 72 displays, for example, "Scene updating", "Scene 2 presenting", etc. Together with the display, the speaker 24 may be used to output audible messages such as "Scene updating", "Scene 2 presenting", etc.

[0086] The operation reception unit 73 receives operations by the user on the sensory device 6, such as operations of the buttons 23 of the control unit 201 of the sensory device 6. The operation reception unit 73 receives re-presentation of a sensory stimulus by button operation.

[0087] The device control unit 74 reads a control program from the device control program storage unit 79 and controls each sensory device 6 according to control signals received from the scene control device 10. The device control program storage unit 79 stores a control program for controlling the tactile stimulation presentation device 37 and the olfactory stimulation presentation device 45 according to control signals. In the case of the visual device 2, the device control unit 74 obtains the mp4 file specified by the scene control device 10 from the storage medium and requests the display device 30 to play it. The display device 30 displays the video on the display 22 by playing the mp4 file. In the case of the hearing device 3, the device control unit 74 retrieves from the storage medium the mp3 file specified by the scene control device 10 and plays it. The earphones 36 output the sound played from the control unit 201. The earphones 36 may have a function for playing mp3 files. In the case of the haptic device 4, the device control unit 74 retrieves the mp3 file specified by the scene control device 10 from the storage medium, and executes the device control program by specifying the vibration intensity level and the mp3 file. The tactile stimulus presentation device 37 vibrates the vibrator 42 under the control of the device control program. The device control unit 74 also executes the device control program by specifying the temperature polarity and temperature level specified by the scene control device 10. The tactile stimulus presentation device 37 controls the temperature of the Peltier element 43 under the control of the device control program. In the case of the olfactory device 5, the device control unit 74 executes a device control program by specifying the cartridge number and intensity specified by the scene control device 10. The olfactory stimulation presentation device 45 outputs a specified scent at a specified intensity under the control of the device control program.

[0088] <Editing the presented stimulus information> The administrator can directly edit the project file for the presentation stimulus information as shown in FIG. 13. On the other hand, for example, the administrator may be able to specify in more detail the timing of presentation of each sensory stimulus for each scene. For this purpose, for example, a description format in which the timing and type of presentation of each sensory stimulus are associated with a timeline for each scene can be considered. When a user performs an operation for scene transition, the sensory stimulus is presented according to the editing content of the switched scene. The scene control device 10 generates a control signal from the presentation stimulus information associated with the timeline and transmits it to the sensory device 6. The sensory device 6 performs control according to the control signal and presents the sensory stimulus. An example of this editing method will be described with reference to FIG. 15.

[0089] 15 is an example of a setting screen 701 for setting presentation stimulus information in association with a timeline. The setting screen 701 displays an area 702 for playing and displaying content, and a play button 703A for playing the content. The area 702 may display a moving image that serves as a guide for setting the presentation stimulus information, or may be empty if the user only wishes to control the timing of the presentation stimulus on the timeline.

[0090] The setting screen 701 also displays a gauge 703C indicating the time (time axis) elapsed from the beginning of the content, and a slider 703B indicating the current playback position of the video displayed in the area 702. In the example of Fig. 15, the left end of the gauge 703C indicates the beginning position of the video, and the further to the right it is, the farther the playback position is from the beginning position. The slider 703B can be moved on the gauge 703C by the operator, and the setting screen plays the content and displays it in the area 702 at a playback position according to the position of the slider 703B on the gauge 703C.

[0091] The setting screen 701 displays areas 711A, 712A, 713A, and 714A for setting presentation stimulus information for each of the senses of vision, hearing, touch, and smell in an area 710 along a gauge 703C that is the time axis of the moving image. Presentation stimulus information designated by the administrator is set in each of the areas 711A, 712A, 713A, and 714A.

[0092] 15, scales are displayed at predetermined time intervals in each of the areas 711A, 712A, 713A, and 714A. The setting terminal 20 accepts settings of presentation stimulus information to be presented by each sensory device 6 according to the playback position of the content, in response to an operation by an operator on each of the areas 711A, 712A, 713A, and 714A.

[0093] Furthermore, the setting screen 701 displays a line 716 in red or the like, which indicates the current playback position of the content displayed in the area 702, superimposed on the area 710. While the content is being played back, the setting screen 701 displays the slider 703B and the line 716 moving to the right in accordance with the transition of the playback position. This allows, for example, the operator to set presentation stimulus information to be presented by each sensory device 6 at any timing (playback position) while visually checking the displayed content.

[0094] 15, in area 714A for setting the sense of smell, a section 719 from playback position 717 to playback position 718 of the content has been specified by the administrator, and therefore setting screen 701 displays section 719 in an emphasized manner. Note that setting screen 701 may accept the specification of section 719 by accepting a drag-and-drop operation. Also, setting screen 701 may accept the specification of section 719 by accepting a swipe operation on a touch panel.

[0095] In the thus set section 719, the administrator can click the mouse or tap the touch panel to set the presentation stimulus information to be presented in the section 719. For example, a pull-down menu for setting olfactory presentation stimulus information is displayed in the section 719 by the administrator's operation.

[0096] In FIG. 16(a), numbers 1 to 6 that can be set as cartridge numbers are displayed in a pull-down menu 751. Similarly, in FIG. 16(b), numerical values ​​0 to 100 that can be set as fragrance intensity are displayed in increments of 10 in a pull-down menu 752. Increments of 10 are just an example, and increments of 1 or 5 may also be used. The administrator can set olfactory presentation stimulus information in section 719 by selecting a cartridge number or fragrance intensity from pull-down menus 751 and 752. The presentation stimulus information that the administrator has set in section 719 may be displayed in a pop-up when, for example, the user hovers the mouse over section 719. The pull-down menu is just an example, and a list of options may be selected by checking a check box.

[0097] Similarly, in section 723 set for the sense of touch, the administrator can click the mouse or tap the touch panel to set the presented stimulus information to be presented in section 723. For example, a pull-down menu for setting presented stimulus information for the sense of smell is displayed in section 723 by the administrator's operation.

[0098] In Fig. 17(a), five mp3 files that can be set as tactile vibration patterns are displayed in a pull-down menu 761. Similarly, in Fig. 17(b), numerical values ​​1 to 7 that can be set as vibration intensity levels are displayed in a pull-down menu 762. In Fig. 17(c), cold, warm, and none that can be set as tactile temperature polarities are displayed in a pull-down menu 763. In Fig. 17(d), numerical values ​​1 to 4 that can be set as temperature levels are displayed in a pull-down menu 764. The administrator can set tactile presentation stimulus information for section 723 by selecting an audio file (vibration pattern), vibration intensity level, temperature polarity, and temperature level from the pull-down menus 761 to 764.

[0099] For the visual and audio senses, the files that can be set may be displayed in a pull-down menu, as in Fig. 17(a). Alternatively, the administrator may be able to input a file name. The administrator may select or specify a video file or an audio file.

[0100] The minimum time for which a section can be set is a design matter. For example, if a piece of content is 24 hours, 10 seconds, 30 seconds, 1 minute, etc. may be set as the minimum section. For more precise settings, it may be possible to specify in milliseconds or frames. Also, if a piece of content is a day, month, year, etc., the appropriate minimum section length may be determined for each.

[0101] <Action or Processing> Next, referring to Fig. 18, the flow of the process of presenting a sensory stimulus according to a scene number will be described. Fig. 18 is a sequence diagram showing the process in which the scene control device 10 requests each sensory device 6 to present a sensory stimulus according to a scene number designated by the user. It is assumed that the IP addresses of the scene operation device 1 and each sensory device 6 are registered in the scene control device 10. It is also assumed that the IP address of the scene control device 10 is registered in the scene operation device 1 and each sensory device 6. It is assumed that the scene operation device 1 and each sensory device 6 have been successfully authenticated by logging in to the scene control device 10, and the scene operation device 1 and the scene control device 10, and the scene control device 10 and the sensory device 6 communicate by specifying the destination, for example, by an IP address.

[0102] S1: First, the manager selects a project file to be executed for the scene control device 10 and presses the communication start button 301. This starts the sensory stimulus presentation process, and the scene control device 10 goes into a standby state for a scene number. Note that the sensory stimulus presentation process does not have to be started by an instruction from the manager of the scene control device 10. For example, the sensory stimulus presentation process may be started by a notification from the scene operation device 1 or another sensory device 6.

[0103] S2: The communication unit 61 of the scene control device 10 transmits to the scene operation device 1 a notification that reception of a scene number has started (that is, that the sensory stimulus presentation process has started).

[0104] S3 to S6: Furthermore, if the sensory device 6 has a scene selection function, the communication unit 61 of the scene control device 10 transmits a prohibition of scene change to each sensory device 6. This is because, in terms of experience design, it is often preferable for each user to experience different sensory stimuli for the same scene. Due to the prohibition of scene change from the communication unit 61, the sensory device 6 cannot accept an operation to change the scene. Therefore, each of the sensory devices 6 presents a sensory stimulus for the same scene instructed by the scene control device 10.

[0105] S7: Having received the notification that reception of scene numbers has begun, the scene operation device 1 may become able to select a scene. This can prevent the scene control device 10 from not responding even if the user selects a scene number and transmits it to the scene control device 10. The display control unit 52 may display on the display 12 that it is now possible to select a scene. After checking the display 12, the user of the scene operation device 1 specifies a scene number by rotating and pressing the button 13. The operation reception unit 53 of the scene operation device 1 accepts the scene number.

[0106] S8: The communication unit 51 of the scene operation device 1 transmits the accepted scene number to the scene control device 10.

[0107] S9: Since the communication section 61 of the scene control device 10 has received the scene number, the scene control section 65 acquires presentation stimulus information corresponding to the scene number from the presentation stimulus information management data storage section 68.

[0108] S10: The scene control unit 65 interprets the presented stimulus information based on the format data and creates a control signal. For the visual device 2, the scene control unit 65 uses video file (.mp4) information as a control signal. For the auditory device 3, the scene control unit 65 uses audio file (.mp3) information as a control signal. For the tactile device 4, the scene control unit 65 uses the audio file (.mp3) and vibration intensity level information as a control signal for the transducer 42, and uses temperature polarity and temperature level information as a control signal for the Peltier element 43. For the olfactory device 5, the scene control unit 65 uses the cartridge number and intensity information as a control signal.

[0109] S11: The communication unit 61 of the scene control device 10 transmits file name information as a control signal to the visual device 2. The communication unit 61 may transmit the file data itself or the scene number to the visual device 2. Note that the display control unit 62 of the scene control device 10 may display "The scene number has been switched to XX" or the like before (or immediately after) transmitting the control signal. When the scene number is transmitted, the visual device 2 creates a control signal.

[0110] S12: The communication unit 61 of the scene control device 10 transmits file name information as a control signal to the auditory device 3. The communication unit 61 may transmit the file data itself or the scene number to the auditory device 3. When the scene number is transmitted, the auditory device 3 creates a control signal.

[0111] S13: The communication unit 61 of the scene control device 10 transmits the file name and information on the vibration intensity level as a control signal for the vibrator 42, and the temperature polarity and temperature level information as a control signal for the Peltier element 43 to the haptic device 4. The communication unit 61 may transmit the file itself or the scene number to the haptic device 4.

[0112] S14: The communication unit 61 of the scene control device 10 transmits the cartridge number and the intensity information to the olfactory device 5 as a control signal.

[0113] S15: The communication unit 71 of the visual device 2 receives the control signal, and the device control unit 74 reads out the video material file specified by the file name from the storage medium and plays it on the display device 30. The display device 30 plays back the material file to play back the video corresponding to the scene number.

[0114] S16: The communication unit 71 of the auditory device 3 receives the control signal, and the device control unit 74 reads out the audio material file specified by the file name from the storage medium and plays it back. The earphones 36 output audio corresponding to the scene number.

[0115] S17: The communication unit 71 of the haptic device 4 receives the control signal, and the device control unit 74 reads out the sound material file specified by the file name from the storage medium, and instructs the vibration intensity level to be executed by the device control program. The device control unit 74 also instructs the temperature polarity and temperature level instructed by the scene control device 10 to be executed by the device control program. Under the control of the device control program, the tactile stimulus presentation device 37 vibrates with the vibration pattern (mp3 file) and vibration intensity level associated with the scene number, and also presents a temperature stimulus with the temperature polarity and temperature level associated with the scene number.

[0116] S18: The communication unit 71 of the olfactory device 5 receives the control signal, and the device control unit 74 executes the device control program by specifying the cartridge number and intensity specified by the scene control device 10. The stimulus presentation unit 202 presents the scent with the type and intensity of the scent associated with the scene number under the control of the device control program.

[0117] In steps S15 to S18, each sensory device 6 receives a control signal, and the display control unit 72 displays "Scene updating", and after the device control unit 74 controls the stimulation presentation unit 202, it may display "Scene XX being presented", etc.

[0118] Thereafter, the multisensory stimulation presentation system 100 repeatedly executes steps S7 to S18 every time the user of the scene operation device 1 selects a new scene number. In this manner, the multisensory stimulation presentation system 100 of the present embodiment allows the user to actively transition scenes and presents sensory stimulations associated with the scenes at each sensory device 6.

[0119] The sensory stimulus presentation process in FIG. 18 may be terminated under the following conditions. A time period previously set in the scene control device 10 has elapsed. Every scene has been played N times (N is 1 or more). The administrator inputs an end command to the scene control device 10.

[0120] 18 ends, the scene control device 10 transmits a notification of the end of scene number acceptance to the scene operation device 1. This may cause the scene operation device 1 to restrict the user's selection of scene numbers. In addition, the scene control device 10 transmits a notification of the lifting of the prohibition on scene changes to the sensory device 6. However, since the sensory device 6 has not been notified of the start of scene number acceptance, the user cannot input a scene number.

[0121] 18, an example in which the user selects one of a plurality of prepared scenes and makes a scene transition is described, but for example, a mechanism may be implemented in which the scene control device 10 records the user's scene transition history and the scene to be transitioned or the selectable scene changes depending on the history. With such a mechanism, a content experience in which the story changes depending on the user can be realized.

[0122] Fig. 19 is a sequence diagram explaining a process in which each user of the sensory device 6 arbitrarily experiences the presentation of a sensory stimulus again. The process in Fig. 19 is executed following, for example, steps S15 to S18, and is executed in a state in which the sensory stimulus presentation process is continuing. Each user of the sensory device 6 can have the same sensory stimulus presented multiple times based on the control signal transmitted by the scene control device 10. In other words, the user can experience the presentation of a stimulus of a certain scene number again (or any number of times if not limited).

[0123] S21 to S24: Each user of the sensory device 6 presses the button 23 of the control unit 201. The operation reception unit 73 receives the pressing.

[0124] S25 to S28: The communication unit 71 of each control unit 201 notifies the scene control device 10 of the button press. This notification of the button press is not necessary. However, by being notified of the button press, the scene control device 10 can count the number of times the button is pressed in each sensory device 6 (the number of times the sensory stimulus is presented). In this case, the scene control device 10 can limit the number of times of playback (the number of times of presentation) to a preset upper limit, so that each user can experience a sense of tension in the case of puzzle-solving content. The upper limit number of times may be variable depending on the sensory device 6. For example, the upper limit number of times of playback for a sensory device 6 that is easy to obtain information from, such as a video, is small, and the upper limit number of times of playback for a sensory device 6 that is difficult to interpret the stimulus, such as the sense of smell (there is a possibility that it is difficult to distinguish or there is a large individual difference) is large. In this way, the multisensory stimulus presentation system 100 can reduce a sense of unfairness due to differences in sensory stimuli.

[0125] In steps S25 to S28, the communication unit 71 of each control unit 201 may transmit the current scene number to the scene control device 10 or transmit a request for a control signal. Such processing is necessary when each sensory device 6 does not hold a control signal.

[0126] S29: When restricting playback, the scene control device 10 transmits a playback restriction request to the sensory device 6 whose playback count has reached the upper limit, and thereafter, until the scene control device 10 transmits a new control signal, the sensory device 6 cannot present a sensory stimulus even when the button 23 is pressed. It is preferable that the restricted sensory device 6 outputs a message such as "Playback is currently restricted" from the display 22 or the speaker 24.

[0127] S30 to S33: The device control unit 74 of the sensory device 6 presents the sensory stimulus again. The sensory device 6 may output a message such as "XXth playback is being performed" from the display 22 or the speaker 24. If an upper limit on the number of playbacks is set, the sensory device 6 may output a message such as "XXth playback remaining" from the display 22 or the speaker 24. The remaining number of playbacks is transmitted from the scene control device 10 to each sensory device 6.

[0128] In this manner, the multisensory stimulation presentation system 100 of the present embodiment can present (play back) to the user a sensory stimulation in an arbitrary scene.

[0129] <<If the user of each sensory device can specify the scene number>> 18, a case has been described in which only the user of the scene operation device 1 can specify the scene number, but the user of each sensory device 6 may also be able to specify the scene number. For example, an administrator sets the scene control device 10 so that the user of each sensory device 6 can specify the scene number.

[0130] 20 is a sequence diagram showing a process in which the scene control device 10 requests each sensory device 6 to present a sensory stimulus according to a scene number designated by the user. Note that the explanation of FIG. 20 may explain differences from FIG. 18.

[0131] S41, S42: This process is similar to steps S1 and S2 in Fig. 18. However, in Fig. 20, the scene control device 10 does not need to notify the scene operation device 1 that it has started accepting scene numbers. If notified, the scene operation device 1 will be able to transmit the scene number, so it is advisable for the scene control device 10 to perform priority control such that the scene number from the scene operation device 1 is given priority (or not) over the scene number selected by the sensory device 6.

[0132] S43 to S46: Unlike in FIG. 18, the communication unit 61 of the scene control device 10 also transmits to each sensory device 6 that it has started accepting scene numbers. Furthermore, the prohibition of scene change is not transmitted to each sensory device 6. If the user of each sensory device 6 can specify a scene number, the visual device 2, auditory device 3, tactile device 4, and olfactory device 5 store the scene number in advance or the scene number is transmitted from the scene control device 10.

[0133] S47: For example, assume that the user of the visual device 2 has specified a scene number. The following process may be similar when the user of another sensory device 6 has specified a scene number. The user of the visual device 2 specifies the scene number by rotating and pressing the button 23. The operation receiving unit 73 of the visual device 2 receives the scene number.

[0134] S48: The communication section 71 of the visual device 2 transmits the accepted scene number to the scene control device 10.

[0135] S49, S50: This process may be the same as steps S9 and S10 in Fig. 18. Therefore, the user of the visual device 2 can actively transition between scenes at his / her own pace and obtain visual information by the presentation of visual stimuli. In this case, the user of each sensory device 6 communicates after obtaining information from his / her own senses in all scenes. Depending on the design of the experience to be provided, it is possible to use cases where each user communicates for each scene.

[0136] S51: The communication unit 61 of the scene control device 10 transmits a control signal including file name information to the visual device 2. Note that the display control unit 62 of the scene control device 10 may display, before (or immediately after) transmitting the control signal, a message such as "Scene number has been switched to XX" in association with each sensory device 6. This allows the administrator to know which scene each sensory device 6 is currently playing.

[0137] S52: The communication unit 71 of the visual device 2 receives the control signal, and the device control unit 74 reads out the video material file specified by the file name from the storage medium and plays it on the display device 30. The display device 30 plays back the video corresponding to the scene number again.

[0138] In this way, the multisensory stimulation presentation system 100 of the present embodiment allows each user to actively transition between scenes for the type of sense that the user is responsible for, and presents sensory stimulation associated with the scene. In addition, the administrator can edit the presentation stimulation information for each scene.

[0139] [Second embodiment] In the first embodiment, one sensory device 6 presents only one sensation, but one sensory device 6 may be capable of presenting a plurality of sensory stimuli.

[0140] Fig. 21 shows a schematic overall view of the system configuration of the multisensory stimulation presentation system 100 of this embodiment. Note that Fig. 21 may be described mainly with respect to the differences from Fig. 2. In Fig. 21, the multisensory stimulation presentation system 100 has a plurality of integrated sensory devices 80. A plurality of users each use the integrated sensory device 80. Each integrated sensory device 80 communicates with the scene control device 10, and each can present a plurality of sensory stimuli.

[0141] Each integrated sensory device 80 has a visual stimulus presentation unit 202, an auditory stimulus presentation unit 202, a tactile stimulus presentation unit 202, and an olfactory stimulus presentation unit 202. For example, the visual stimulus presentation unit 202 is an AR (Augmented Reality) / VR monitor 81, the auditory stimulus presentation unit 202 is an earphone 82, the tactile stimulus presentation unit 202 is a tactile stimulus presentation device 83 that is in close contact with the forehead, and the olfactory stimulus presentation unit 202 is an olfactory stimulus presentation device 84 that is provided in front of the nose. Therefore, each integrated sensory device 80 can present each sensory stimulus. The integrated sensory device 80 is not a wearable type, and may be configured such that, for example, a tactile stimulus presentation unit and an olfactory stimulus presentation unit are included in a rod-shaped device that can be held in one hand, the visual stimulus presentation unit is a small display device attached to the rod-shaped device, and the auditory stimulus presentation unit is an earphone.

[0142] The integrated sensory device 80 can switch which sensory stimuli it can present under the control of the scene control device 10. That is, the integrated sensory device 80 can present any one or more sensory stimuli from among vision, hearing, and smell. When only one sense can be presented, the processing form will be the same as in the first embodiment. Also, a mechanism may be introduced in which the integrated sensory device 80 displays a user input interface (such as a button) for switching the senses, and when the user switches the type of sense to be presented, that information is transmitted to the scene control device 10, and the scene control device 10 can switch which sensory stimulus to present.

[0143] With the integrated sensory device 80, for example, after the implementation of the puzzle-solving content, each user can easily experience a different sensory stimulus. For example, during the implementation of the puzzle-solving content, Person A is in charge of vision, and Person B is in charge of smell. After the implementation of the puzzle-solving content is over, Person A may want to know what the scent was like in scene 3. In such a case, the scene control device 10 removes the restriction on the sensory stimulus presented by the integrated sensory device 80, allowing Person A to experience what the scent was like in scene 3.

[0144] The hardware block diagram of the control unit 201 may be the same as that of Fig. 10, and the hardware block diagram of the stimulation presentation unit 202 (display device 30, earphones 36, tactile stimulation presentation device 37, olfactory stimulation presentation device 45) may be the same as that of Fig. 11. However, in this embodiment, material files for each sensory stimulation are stored in the storage medium mounting section of one control unit 201. Also, one integrated sensory device 80 has all the actuators.

[0145] <About the function> Fig. 22 is a functional configuration diagram showing the functions of the scene operation device 1, the scene control device 10, and the control unit 201 of each sensory device 6 of this embodiment. Note that the explanation of Fig. 22 will mainly focus on the differences from Fig. 12.

[0146] FIG. 22 differs from FIG. 12 in that the control unit 201 of the integrated sensory device 80 has a sensory restriction unit 75. The sensory restriction unit 75 restricts the sensory stimuli that can be presented by the integrated sensory device 80. The types of sensory stimuli that can be presented or that are restricted are notified from the scene control device 10. Alternatively, the user can set the types of sensory stimuli that can be presented or that are restricted in the integrated sensory device 80.

[0147] <Action or Processing> 23 is a sequence diagram showing a process in which the scene control device 10 requests each integrated sensory device 80 to present a sensory stimulus according to a scene number designated by the user. Note that the explanation of FIG. 23 may mainly focus on the differences from FIG. 18.

[0148] S61: The processes in steps S61 and S62 may be similar to those in steps S1 and S2 in FIG.

[0149] S63 to S66: Next, the communication unit 61 of the scene control device 10 assigns a sensory type to each integrated sensory device 80. The communication unit 71 of each integrated sensory device 80 receives the assigned sensory type. It is preferable that the display control unit 72 presents the sensory stimulus of the currently assigned type from the AR / VR monitor 81 or the earphones 82.

[0150] The following method is conceivable as a method for the scene control device 10 to assign a type of sense to the integrated sensory device 80. It is assumed that the scene control device 10 has registered therein identification information, etc. of the four integrated sensory devices 80 to which a type of sense is assigned. Randomly assign senses to each integrated sensory device 80 so that the types of senses presented do not overlap. Information on the type of senses previously assigned to each integrated sensory device 80 is stored, and a type of sense is assigned to each integrated sensory device 80 so as to eliminate variation in the senses presented by the four integrated sensory devices 80. For example, vision is assigned to the integrated sensory device 80 that has been assigned vision as a sensory type the fewest times. The scene control device 10 may obtain from the sensory device 6 the type of senses previously assigned to each integrated sensory device 80. When there is no scent remaining, the integrated sensory device 80 may notify the user, and the scene control device 10 may assign a type of sense other than the sense of smell to the integrated sensory device 80. The integrated sensory device 80 notifies the scene control device 10 of information on the user's desired sense. The scene control device 10 prioritizes the requests, but in the event of overlaps, randomly assigns a type of sense to each integrated sensory device 80 so as not to overlap. The control unit 201 of the integrated sensory device 80 displays the four types of senses and accepts control related to the desired sense. The administrator manually sets the correspondence between the integrated sensory device 80 and the type of sensory provided. In this case, each integrated sensory device 80 notifies the scene control device 10 of the type of sensory set in itself.

[0151] The process after step S3 may be the same as that in the sequence diagram of Fig. 18. Also in this embodiment, each user may be able to present the same sensory stimulus again as in Fig. 19. Also, as in Fig. 20, the user of each integrated sensory device 80 may be able to switch the scene played by the device itself.

[0152] 23, the scene control device 10 may assign multiple sensory stimuli to one integrated sensory device 80. This allows users to experience multiple types of sensory stimuli in each scene, making it possible to solve puzzles even if the number of people (four) required for the number of senses does not gather.

[0153] FIG. 24 is a sequence diagram illustrating a process in which each user of the integrated sensory device 80 experiences the presentation of an arbitrary sensory stimulus in an arbitrary scene.

[0154] S71: When there is no longer a need to restrict the sensory stimuli presented by the integrated sensory device 80 (for example, when the puzzle-solving content has ended), the administrator inputs an operation to the scene control device 10 to lift the restriction on the presentation of the sensory stimuli.

[0155] S72 to S75: The communication unit 61 of the scene control device 10 transmits the restriction lift to each integrated sensory device 80. The sensory restriction unit 75 of each integrated sensory device 80 lifts the restriction on the presentation of sensory stimuli that can be presented. The restriction lift may be performed automatically when the sensory stimulus presentation process ends. In addition, if the scene control device 10 also permits each user to select a scene, it transmits to each sensory device 6 that it has started accepting scene numbers.

[0156] S76: For example, assume that a user in charge of vision specifies a scene number and the type of sense they wish to experience (smell). The following process may be similar when a user of another integrated sensory device 80 specifies a scene number and type of sense. The user of the integrated sensory device 80 (in charge of vision) specifies the scene number and type of sense by rotating and pressing button 23. The operation reception unit 73 of the integrated sensory device 80 receives the scene number and type of sense.

[0157] S77: The communication unit 71 of the visual device 2 transmits the received scene number and type of sense (olfaction) to the scene control device 10.

[0158] S78, S79: This process may be similar to steps S9, S10 in FIG.

[0159] S80: The communication unit 61 of the scene control device 10 transmits a control signal including the cartridge number and intensity as information to the integrated sensory device 80. Note that the display control unit 62 of the scene control device 10 may display, before (or immediately after) transmitting the control signal, a message such as "The scene number has been switched to XX and an olfactory stimulation has been presented" in association with each integrated sensory device 80. This allows the administrator to know which scene each integrated sensory device 80 is currently playing.

[0160] S81: The communication unit 71 of the integrated sensory device 80 receives the control signal, and the device control unit 74 executes the control program by specifying the cartridge number and the intensity. The olfactory stimulation presentation device 45 presents an aroma according to the scene number under the control of the control program.

[0161] In this way, with the integrated sensory device 80, the same user can select not only the scene but also the type of sense, and can experience the presentation of all sensory stimuli associated with the scene.

[0162] <Major Effects> The multisensory stimulation presentation system 100 of this embodiment allows the user to call up each scene with a simple operation and present a sensory stimulation, making it possible to present different sensory stimulations on each sensory device 6 through active scene transitions by the user. In addition, an administrator can edit, store, and present the presentation stimulation information for each sense in each scene of the content.

[0163] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0164] For example, the scene control device 10 may exist in the cloud (Internet). In this case, the scene control device 10 corresponds to a server, and the scene operation device 1 and the sensory device 6 correspond to clients. Therefore, users who operate the scene operation device 1 and the sensory device 6 may be located at different locations. Each user of the sensory device 6 can communicate via an existing online conference system. Each user who receives presentation of sensory stimuli based on the same project file simply needs to log in to the same conference.

[0165] Also, instead of the user of the scene operation device 1 or the user of each sensory device 6 selecting a scene, the scene control device 10 may transmit a control signal for the next scene to the sensory device 6 when a predetermined time for each scene has elapsed. In this case, the user of the sensory device 6 may be able to request the scene control device 10 to postpone the scene transition.

[0166] In addition, each user may be able to operate the scene operation device 1 and the sensory device 6 by voice.

[0167] When the control unit 201 of the sensory device 6 is replaced by a smartphone or the like, an application that controls the sensory device 6 is executed on the smartphone. When the smartphone or the like executes the application, the smartphone or the like has the function of the control unit 201 of the sensory device.

[0168] In addition, in this embodiment, one sensory device 6 presents one sensory stimulus, but when using the integrated sensory device 80, it is also possible to limit the presentation of one sensory stimulus to only one integrated sensory device 80. For example, when the vision of the integrated sensory device 80 is limited, the user can experience the situation of a visually impaired person.

[0169] There may also be a sensory device 6 that presents a taste stimulus, and the integrated sensory device 80 may present a taste stimulus as the sensory stimulus.

[0170] In addition, the configuration example in Fig. 12 and the like is divided according to main functions in order to facilitate understanding of the processing by the scene operation device 1, the scene control device 10, and the control unit 201 of each sensory device 6. The present invention is not limited by the manner in which the processing units are divided or the names of the processing units. The processing by the scene operation device 1, the scene control device 10, and the control unit 201 of each sensory device 6 can be divided into even more processing units depending on the processing content. Also, it can be divided so that one processing unit includes even more processes.

[0171] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0172] Embodiments of the present invention provide significant improvements in computer power and functionality. These improvements allow users to utilize computers that provide more efficient and robust interaction with tables, which are ways of storing and presenting information in information processing devices. Furthermore, embodiments of the present invention provide a better user experience through the use of more efficient, powerful and robust user interfaces. Such user interfaces provide better interaction between humans and machines.

[0173] <Additional Notes> [Appendix 1] The information processing device includes: A presentation stimulus information management data storage unit is provided in which information on visual stimuli for the visual device, information on auditory stimuli for the auditory device, information on tactile stimuli for the tactile device, and information on olfactory stimuli for the olfactory device are stored in association with identification information for each scene; When the scene identification information is received, a control signal based on the visual stimulus information associated with the scene identification information is transmitted to the visual device, a control signal based on the auditory stimulus information associated with the scene identification information is transmitted to the auditory device, a control signal based on the tactile stimulus information associated with the scene identification information is transmitted to the tactile device, and a control signal based on the olfactory stimulus information associated with the scene identification information is transmitted to the olfactory device. [Appendix 2] The information processing device includes: displaying information on visual stimuli for the visual device, information on auditory stimuli for the auditory device, information on tactile stimuli for the tactile device, and information on olfactory stimuli for the olfactory device, which are associated with identification information of each scene; Editing of the visual stimulus information, the auditory stimulus information, the tactile stimulus information, or the olfactory stimulus information is accepted. [Appendix 3] The visual stimulus information, the auditory stimulus information, the tactile stimulus information, or the olfactory stimulus information is associated with a timeline of the content, and editing of the visual stimulus information, the auditory stimulus information, the tactile stimulus information, or the olfactory stimulus information is accepted for a section of the timeline specified by the user. [Appendix 4] The integrated sensory device presents only the sensory stimuli permitted by the information processing device. [Explanation of symbols]

[0174] 1 Scene operation device 2. Visual Devices 3. Hearing Devices 4. Haptic Devices 5. Olfactory Device 10 Scene Control Device 100 Multisensory Stimulation Presentation System

Claims

1. A multisensory stimulation presentation system that presents a sensory stimulation corresponding to content to a user, An information processing device; A visual device that presents only visual stimuli; an auditory device that presents only auditory stimuli; A tactile device that presents only tactile stimuli; an olfactory device that presents only olfactory stimuli; the information processing device accepts a designation of a scene of the content, and requests the visual device, the auditory device, the tactile device, and the olfactory device to present the sensory stimulus associated with the scene; A multisensory stimulus presentation system, characterized in that the visual device presents visual stimuli corresponding to the scene, the auditory device presents auditory stimuli corresponding to the scene, the tactile device presents tactile stimuli corresponding to the scene, and the olfactory device presents olfactory stimuli corresponding to the scene.

2. a scene operation device that accepts the scene designation, The scene operation device holds identification information of the scene, The multisensory stimulation presentation system according to claim 1 , further comprising: a display configured to receive a selection of identification information of the scene displayed on the display; and transmit the identification information of the scene to the information processing device.

3. The multisensory stimulation presentation system of claim 1 or 2, wherein the visual device, the auditory device, the tactile device, and the olfactory device have a display, and information about the scene in which a visual stimulus, an auditory stimulus, a tactile stimulus, or an olfactory stimulus is presented is displayed on the display.

4. The multisensory stimulation presentation system of claim 3, characterized in that the visual device, the auditory device, the tactile device, and the olfactory device have buttons, and when the button is pressed, the visual stimulus, auditory stimulus, tactile stimulus, or olfactory stimulus corresponding to the scene are presented again.

5. the visual device, the auditory device, the tactile device, or the olfactory device transmits to the information processing device a notification that the button has been pressed; The multisensory stimulation presentation system according to claim 4, characterized in that, when the upper limit of the number of playbacks set for each sensory stimulus is reached, the information processing device requests the visual device, the auditory device, the tactile device, or the olfactory device that has reached the upper limit to limit the presentation of the sensory stimulus.

6. the information processing device transmits to the visual device, the auditory device, the tactile device, and the olfactory device a notification that the scene change is prohibited; The multisensory stimulation presentation system according to claim 1 , wherein the visual device, the auditory device, the tactile device, and the olfactory device do not accept an operation to change the scene, and each presents the sensory stimulation in the same scene.

7. the visual device, the auditory device, the tactile device, and the olfactory device each hold identification information of the scene and are capable of receiving designation of the scene; the visual device, the auditory device, the tactile device, or the olfactory device transmits the received identification information of the scene to the information processing device; The multisensory stimulation presentation system of claim 1 , wherein any one of the visual device, the auditory device, the tactile device, and the olfactory device that transmitted the scene identification information presents the sensory stimulation corresponding to the scene identification information requested by the information processing device.

8. A multisensory stimulation presentation system that presents a sensory stimulation corresponding to content to a user, An information processing device; A plurality of integrated sensory devices each including a stimulus presentation unit that presents a visual stimulus, a stimulus presentation unit that presents an auditory stimulus, a stimulus presentation unit that presents a tactile stimulus, and a stimulus presentation unit that presents an olfactory stimulus; having the information processing device receives a designation of a scene of the content, and requests the integrated sensory device to present a visual stimulus, an auditory stimulus, a tactile stimulus, and an olfactory stimulus among the sensory stimuli associated with the scene, separately; A multisensory stimulus presentation system characterized in that when requested to present the visual stimulus, the integrated sensory device presents a visual stimulus corresponding to the scene, when requested to present the auditory stimulus, the integrated sensory device presents an auditory stimulus corresponding to the scene, when requested to present the tactile stimulus, the integrated sensory device presents a tactile stimulus corresponding to the scene, and when requested to present the olfactory stimulus, the integrated sensory device presents an olfactory stimulus corresponding to the scene.

9. The integrated sensory device is limited in the types of sensory stimuli that it can present; The information processing device transmits a lift of the restriction to the integrated sensory device in a predetermined case; the integrated sensory device accepts an operation to present any one of the sensory stimuli, and requests a control signal from the information processing device for presenting a sensory stimulus corresponding to the scene; The multisensory stimulation presentation system according to claim 8 , wherein any one of the sensory stimulations is presented based on the control signal transmitted from the information processing device.

10. Multiple users can use it individually. A visual device that presents only visual stimuli; A hearing device that presents only auditory stimuli and A tactile device that presents only tactile stimuli An information processing device that communicates with an olfactory device that presents only olfactory stimuli and presents sensory stimuli according to content, accepting a designation of a scene of the content, and requesting the visual device, the auditory device, the tactile device, and the olfactory device to present the sensory stimulus associated with the scene; An information processing device characterized in that the visual device presents visual stimuli corresponding to the scene, the auditory device presents auditory stimuli corresponding to the scene, the tactile device presents tactile stimuli corresponding to the scene, and the olfactory device presents olfactory stimuli corresponding to the scene.

Images