Image display method, program, and image display system

Abstract

To provide video display which gives a user a sense of immersion and presence.SOLUTION: A sensor 20 that detects a motion of a user, display devices 40-1 and 40-2 that display an image in a video space, and a processing device 11 that determines, on the basis of an output of the sensor 20, whether a first motion of the user after a stimulus to a tactile sense of the user is presented satisfies a first condition are included, and the processing device 11 causes the display devices 40-1 and 40-2 to display a first image in a first region of the video space when it is determined that the first motion satisfies the first condition.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an image display method, a program, and an image display system. [Background technology]

[0002] In a system for displaying video, a technique is known in which the video changes in response to the user's movements on the display screen in order to give the user a sense of immersion and realism. For example, Patent Document 1 discloses a video display system that displays two-dimensional video from a camera position on a display screen, in which the camera position is moved and image objects appear in the video in response to the user's position on the display screen. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-219820 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology disclosed in Patent Document 1, the trigger for a change in the image is the movement of the user viewing the image or the movement of the camera. Therefore, unless the user consciously moves on the display screen, the process of making the image object appear in the image may not proceed. In other words, unless the user consciously moves on the display screen, the image object will not appear. In this sense, there is a problem in that the sense of immersion is lacking and the sense of realism is reduced. [Means for solving the problem]

[0005] One aspect of an image display method according to the present disclosure includes presenting a stimulus to a user's tactile sense, determining whether a first action of the user after presenting the stimulus satisfies a first condition based on the output of at least one sensor that detects the user's action, and, if it is determined that the first action satisfies the first condition, displaying a first image in a first region of a video space by at least one display device.

[0006] One aspect of the program of the present disclosure causes a computer to perform the following steps: determine whether a first action of a user after presenting a stimulus to the user's tactile sense satisfies a first condition based on the output of at least one sensor that detects the user's action; and, if it is determined that the first action satisfies the first condition, display a first image in a first region of a video space on at least one display device.

[0007] One aspect of an image display system according to the present disclosure includes at least one sensor that detects a user's actions, at least one display device that displays an image in a video space, and an information processing device that performs the following operations: determining whether a first action of the user after presenting a tactile stimulus to the user satisfies a first condition based on the output of the at least one sensor; and, if it is determined that the first action satisfies the first condition, causing the at least one display device to display a first image in a first area of ​​the video space. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram of an image display system according to a first embodiment. [Figure 2] FIG. 2 is a functional block diagram showing an example of functions of the processing device according to the first embodiment. [Figure 3] FIG. 2 is a diagram showing an example of the arrangement of elements in the image display system according to the first embodiment. [Figure 4] 10A and 10B are diagrams illustrating an example of a background video and an image object superimposed on the background video. [Figure 5] FIG. 10 is a diagram showing the state in which the tactile stimulation device is in operation. [Figure 6] FIG. 10 is a diagram showing a state in which the user has changed the direction of his or her head. [Figure 7] 10A and 10B are diagrams showing how the display mode of an image object changes. [Figure 8] FIG. 10 is a diagram showing a state in which a user approaches an image object. [Figure 9] 10A and 10B are diagrams showing further changes in the display mode of the image object. [Figure 10] 10 is a flowchart showing an example of a first processing operation of the processing device. [Figure 11] 10 is a flowchart showing an example of a second processing operation subsequent to a first processing operation of the processing device. [Figure 12] FIG. 10 is a configuration diagram of an image display system according to a second embodiment. [Figure 13] FIG. 10 is a functional block diagram showing an example of functions of a processing device according to a second embodiment. [Figure 14] FIG. 14 is a perspective view showing the appearance of the MR glasses of FIG. [Figure 15] FIG. 10 is a diagram showing an example of the arrangement of elements in an image display system according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions, and some parts are shown schematically to facilitate understanding. Furthermore, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited thereto.

[0010] 1. First embodiment 1.1. Image display system overview Fig. 1 is a configuration diagram of an image display system 1 according to the first embodiment. Below, an overview of the image display system 1 according to the first embodiment will be described with reference to Fig. 1. The image display system 1 includes a computer 10, a sensor 20, a tactile stimulation device 30, display devices 40-1 and 40-2, and a sound generation device 50.

[0011] The computer 10 controls each unit in the image display system 1. The computer 10 includes a processing device 11, a storage device 12, an input IF (interface) 13, and an output IF 14. These devices are electrically connected to each other via, for example, a bus Bs.

[0012] The processing device 11 controls the operation of the computer 10. The processing device 11 includes one or more CPUs. However, the processing device 11 may include an FPGA, a DSP, an ASIC, etc. instead of or in addition to a CPU. Here, CPU is an abbreviation for Central Processing Unit, FPGA is an abbreviation for Field-Programmable Gate Array, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit.

[0013] The storage device 12 is configured to include, for example, a volatile memory such as RAM and a non-volatile memory such as ROM. Here, RAM is an abbreviation for Random Access Memory, and ROM is an abbreviation for Read Only Memory. The storage device 12 stores a control program 121 for the processing device 11 to control each unit, video and audio data 122 consisting of video data and audio data, control data 123 including control parameters for the tactile stimulation device 30, etc. The volatile memory of the storage device 12 is used by the processing device 11 as a work area for the processing device 11.

[0014] The storage device 12 may be configured by combining multiple types of recording media. The storage device 12 may also be a portable recording medium that is detachable from the computer 10. A part or all of the storage device 12 may be provided in an external storage device, an external server, or the like. A part or all of the various information stored in the storage device 12 may be stored in advance in the storage device 12, or may be obtained from an external storage device, an external server, or the like.

[0015] The input IF 13 and the output IF 14 are hardware that function as transmitting / receiving devices for communicating with other devices. The input IF 13 and the output IF 14 are also referred to as, for example, a network device, a network controller, a network card, a communication module, etc. The input IF 13 and the output IF 14 may include a connector for wired connection and an interface circuit corresponding to the connector. The input IF 13 and the output IF 14 may also include a wireless communication interface. Examples of connectors and interface circuits for wired connection include products that comply with wired LAN, IEEE 1394, USB, etc. Examples of wireless communication interfaces include products that comply with wireless LAN, Bluetooth (registered trademark), etc. Here, LAN is an abbreviation for Local Area Network, and USB is an abbreviation for Universal Serial Bus.

[0016] The input IF13 is a device for inputting the imaging data Dt output from the sensor 20. The output IF14 is a device for supplying control data Td to the tactile stimulation device 30, supplying a video signal Vd-1 to the display device 40-1, supplying a video signal Vd-2 to the display device 40-2, and supplying sound data to the sound generation device 50.

[0017] The sensor 20 is, for example, a photographing device that photographs the participant viewing the video and audio from the image display system 1 and outputs photographing data Dt indicating the photographing results to the computer 10. The photographing device preferably has a sensitivity sufficient to distinguish the participant's facial expression in a dark place, and an infrared camera may be used.

[0018] A stereo camera may also be used as the sensor 20. By using a stereo camera, the distance and direction from the sensor 20 to the user can be detected more accurately. A distance sensor may also be used instead of or in addition to the image capture device. Examples of distance sensors include an ultrasonic sensor, a range sensor, and a millimeter-wave radar.

[0019] An acceleration sensor may also be used as the sensor 20. The acceleration sensor is attached to the head of the user, for example, and can detect changes in the user's behavior, particularly changes in head movement.

[0020] The sensor 20 may also be a sensor that detects the biological information of the user. Examples of sensors that detect the biological information of the user include an electroencephalogram (EEG) sensor, a pulse wave sensor, and a heart rate sensor. The EEG sensor is worn on the user's head, and may be any sensor with sufficient accuracy to determine whether the user is in a tense or relaxed state. The pulse wave sensor is worn on the user's wrist, for example. A sensor included in a smartwatch worn by the user may be used as the pulse wave sensor. The EEG sensor is also called a heart rate sensor.

[0021] For the sake of simplicity, the present embodiment uses one image capturing device as the sensor 20, but two or more sensors 20 may be used. Furthermore, the sensor 20 may be a distance sensor, an acceleration sensor, or a biometric sensor, or a combination of these sensors, instead of or in addition to the image capturing device. When these sensors are used in combination, the outputs of the sensors may be weighted or prioritized.

[0022] The experiencer refers to a person who views images and sounds from the image display system 1. Hereinafter, for convenience, the experiencer will be referred to as a user.

[0023] The tactile stimulation device 30 is a device that presents a stimulation to the user's tactile sense. Presenting a stimulation to the user's tactile sense includes stimulating the user's tactile sense. Presenting a stimulation to the user's tactile sense also includes transmitting a tactile sensation to the user. The tactile stimulation device 30 includes at least one of a blower, a vibrator, and a driver.

[0024] The blower includes at least one of a blower fan and a vortex generator.

[0025] The blower fan blows air towards the user, thereby awakening the user. When a state changes from no wind to a state where the user is being blown by air, the user's sense of touch is stimulated, which may awaken the user. Alternatively, the blower fan may be stopped from blowing air from the state where the user is being blown, thereby changing the state inside the audiovisual room 70 to a no wind state. In essence, it is sufficient to change the movement of air inside the audiovisual room 70. When the blower fan is stopped, the state changes from a state where the user is being blown by air to a no wind state, and the stimulation to the sense of touch disappears, which may awaken the user.

[0026] The blower fan may stimulate the user's tactile sense by swinging a string hanging from the ceiling of the audiovisual room 70. It is preferable that the blower fan is configured so that the strength and direction of the wind can be freely changed.

[0027] A vortex ring generator generates a vortex ring. The vortex ring generator is a so-called "air cannon." The vortex ring generated by the vortex ring generator is sent into the air. The vortex ring can travel a relatively long distance without dissipating, allowing it to be accurately aimed at the user. When the vortex ring hits the user's hand, face, etc., it stimulates the user's tactile sense, which may cause the user to become aware of it.

[0028] It is preferable that the vortex ring generator be configured to be able to freely change the direction of the vortex ring. The vortex ring generator may also use heated air to generate warm vortex rings. The vortex ring generator may also use an air freshener, aroma oil, or the like to generate scented vortex rings. The vortex ring generator may also use a smoke machine to generate smoky vortex rings to provide a visual effect to the user.

[0029] The vibration device is placed, for example, under a specific position on the floor of the audiovisual room 70. A vibration motor, a vibrator, or the like is used as the vibration device. When a user is standing at a specific position, the vibration device vibrates the floor, stimulating the user's tactile sense and potentially awakening the user. In addition, for a wheelchair user, the vibration device may be placed on the wheelchair so that the seat of the wheelchair is vibrated.

[0030] The drive device is placed, for example, below a specific position on the floor of the audiovisual room 70. A device that combines a motor and a cam is used as the drive device. The drive device displaces the floor surface by slightly moving a portion of the floor surface up and down. When a user is standing at a specific position, the drive device displaces a portion of the floor surface, stimulating the user's tactile sense and potentially awakening the user's awareness.

[0031] The drive device may also be placed on the wall of the audiovisual room 70. The drive device slightly protrudes a portion of the wall. When a user is standing in a specific position, the drive device causes the portion of the wall to protrude, so that the portion of the wall comes into contact with the user. This stimulates the user's tactile sense and may cause the user to become aware of the movement.

[0032] In this way, presenting tactile stimuli involves, for example, moving the air in the video space with a fan. For example, when wind starts to blow, the user may take action such as turning their face in the direction from which the wind is blowing, i.e., into the wind. Therefore, moving the air with a fan is equivalent to giving the user awareness.

[0033] Furthermore, presenting a tactile stimulus may involve, for example, bringing a thread in the video space into contact with the user using a fan. For example, if a thread hanging from the ceiling sways in the wind and touches the user's head, the user may take action such as looking around. Other examples include cloth, feathers, or paper. Therefore, bringing the first object into contact with the user using a fan corresponds to giving the user an awareness.

[0034] Furthermore, presenting a stimulus to the tactile sense may involve, for example, vibrating a portion of the floor surface that the user is touching using a vibration device. For example, when the user senses the vibration on the floor surface, the user may take action such as looking down. As another example, a structure constituting the video space, such as a wall or a screen, or an object installed in the video space, such as a board or a ball, may be used. Therefore, vibrating the second object that the user is touching using a vibration device corresponds to giving the user an awareness.

[0035] Furthermore, presenting a tactile stimulus may involve, for example, touching a part of a wall surface in the image space to the user using a drive device. For example, when a part of the wall surface touches the user, the user may take action, such as gazing at the contacted part. Other examples include a screen, cloth, thread, feathers, etc. Therefore, touching a third object to the user using a drive device corresponds to giving the user an awareness.

[0036] Specifically, the display devices 40-1 and 40-2 are projectors. The display device 40-1 projects an image based on a video signal Vd-1 supplied from the computer 10. Like the display device 40-1, the display device 40-2 projects an image based on a video signal Vd-2 supplied from the computer 10. Note that, for convenience, two display devices, 40-1 and 40-2, are shown in FIG. 1, but the number of display devices may be one, or three or more. Furthermore, the display devices 40-1 and 40-2 are not limited to projectors and may be direct-view panel display devices.

[0037] The sound generation device 50 includes a sound source device 51 and speakers 52-1 to 52-2. The sound source device 51 applies a predetermined effect to the sound data Ad supplied via the output IF 14 and converts it into n-channel analog sound signals Ad-1 to Ad-n. The sound source device 51 amplifies each of the sound signals Ad-1 to Ad-n and supplies the amplified sound signals Ad-1 to Ad-n to the speakers 52-1 to 52-n in one-to-one correspondence, respectively.

[0038] The effects that the sound source device 51 imparts to the audio data include a stereophonic effect that imparts a three-dimensional sound spread to background sounds based on BGM data among the audio data, and an effect that fixes or moves the sound image position of a specific sound based on specific audio data. The speakers 52-1 to 52-n are sound emitting devices that convert the audio signals Ad-1 to Ad-n into sounds, which are physical vibrations, and output them into the air. In Fig. 1, n speakers 52-1 to 52-n are shown, one for each channel. However, if the sound image position is not changed and a stereophonic effect is not imparted, the number of speakers may be one.

[0039] 1.2. Functions of the Processing Unit

[0040] Fig. 2 is a functional block diagram showing an example of the functions of the processing device 11 according to the first embodiment. Fig. 2 shows a functional configuration constructed when the processing device 11 of the computer 10 executes a control program 121 stored in the storage device 12. As shown in Fig. 2, the processing device 11 has functions as a processing control unit 111, a video processing unit 112, a tactile stimulation processing unit 113, a sound processing unit 114, and a determination unit 115.

[0041] The processing control unit 111 starts reading out the audio and video data 122 from the storage device 12 through a predetermined operation or the like, and controls the video processing unit 112, tactile stimulation processing unit 113, audio processing unit 114, and determination unit 115.

[0042] The video processing unit 112 generates video signals Vd-1 and Vd-2 based on the video data in the audio and video data 122 read by the processing control unit 111 under the control of the processing control unit 111. The video signal Vd-1 generated by the video processing unit 112 is supplied to the display device 40-1 via the output IF14, and the video signal Vd-2 is supplied to the display device 40-2 via the output IF14.

[0043] The tactile stimulus processing unit 113 generates a control signal Cd based on the control data 123 read by the processing control unit 111, under the control of the processing control unit 111. The control signal Cd generated in the tactile stimulus processing unit 113 is supplied to the tactile stimulus device 30 via the output IF14.

[0044] The audio processing unit 114 outputs audio data Ad from the audio-visual data 122 read by the processing control unit 111. The audio data Ad output from the audio processing unit 114 is supplied to the sound generating device 50 via the output IF 14.

[0045] The determination unit 115 analyzes the captured data Dt captured by the sensor 20 in accordance with instructions from the process control unit 111, and determines whether there has been a change in the user's behavior, in other words, whether the user's first action satisfies a first condition. Specifically, the user's behavior refers to at least one of the orientation of the face and body, facial expression, gaze direction, standing position, etc. The speed of these changes over time may be taken into consideration when determining changes in the user's behavior. Specifically, if the speed of change in behavior is less than a threshold, it may be determined that there is no change in behavior.

[0046] The first action is a user's action, behavior, or action. Specifically, the first action is a user looking around, a user moving within the audiovisual room 70, a user stopping in a specific area, etc. A user who is made aware by the tactile stimulation may look around, or may move in the direction of the tactile stimulation. A user who is made aware by the tactile stimulation may also stop in place.

[0047] The first condition includes a behavioral pattern that a user who has been awakened by the tactile stimulation may take. More specifically, the behavioral pattern is preferably one that is likely to result in the user discovering the first image displayed in the first area. For example, the first condition may include the user facing the direction of the first area and viewing the image displayed in the first area. The first area is an area that displays the first image. For example, a user who has been awakened by the tactile stimulation may look around as a result of the awakening, and therefore the user may turn toward the first area. If the user faces the direction of the first area, the user will be more likely to discover the first image that is about to be displayed.

[0048] In this embodiment, display devices 40-1 and 40-2 display an image in which image objects are superimposed on a background image. The image data includes background image data representing the background image and image object data representing the image objects. The background image data is image data representing a background image such as a forest, ocean, sky, or space, and in this embodiment, an image of a forest containing multiple trees is used as the background image. The image object data is data representing image objects such as animals, birds, fish, people, and buildings that are superimposed on the background image. Image objects include images of real objects, images of virtual objects, etc., but do not include backgrounds. In this embodiment, data representing houses and people is used as the image object data.

[0049] The sound data includes background music (BGM) data and sound effect data. The background music data is data representing music or environmental sounds played to create an atmosphere when displaying video. The sound effect data is data representing sound effects played in response to changes in the movement of image objects for a user viewing the video. Sound effects include, for example, the sound of a person walking, the sound of a door opening and closing, the sound of leaves rustling in the wind, etc. Furthermore, the sound effects may be pre-recorded natural sounds or artificial sounds synthesized using electronic musical instruments, etc.

[0050] When an operator or an external device issues an instruction to start display, the processing control unit 111 starts reading out the video and audio data 122. Of the read video and audio data 122, the processing control unit 111 supplies the video data to the video processing unit 112 and the audio data to the audio processing unit 114.

[0051] The video processor 112 generates, as video signals Vd-1 and Vd-2, a video in which an image object based on the image object data is superimposed on a background video based on the background video data out of the video data.

[0052] 1.3. Example of Image Display System 1 layout Fig. 3 is a diagram showing an example of the arrangement of each element in the image display system 1 according to the first embodiment. Note that each element in the image display system 1 is provided in an audiovisual room 70, but in Fig. 3, in order to easily explain the example arrangement, part of the ceiling and wall surfaces in the audiovisual room 70 are omitted, and only speaker 52-1 of speakers 52-1 to 52-n is shown.

[0053] The sensor 20 is attached to, for example, wall surface 70-2. The tactile stimulation device 30 is attached to, for example, wall surface 70-2. The display device 40-1 projects an image onto wall surface 70-1 in the audiovisual room 70. The display device 40-2 projects an image onto wall surface 70-2 in the audiovisual room 70. The speaker 52-1 is attached to, for example, wall surface 70-2. In this example, images are projected onto walls 70-1 and 70-2, but the projection surface for the images may be all of the walls of the audiovisual room 70, or may include at least one of the ceiling and floor surfaces.

[0054] 1.4. Specific Operation of Image Display System 1 Next, specific operations of the image display system 1 will be described with reference to FIGS. 4 to 9. FIG. 4 is a diagram illustrating an example of a background image and an image object superimposed on the background image. As shown in FIG. 4, forest images are continuously played back as background images 91-1 and 91-2. Specifically, background image 91-1 is projected onto wall surface 70-1 by display device 40-1, and background image 91-2 is projected onto wall surface 70-2 by display device 40-2. In this example, background images 91-1 and 91-2 are images of a dimly lit forest. User U is standing in first specific area SA1 facing wall surface 70-1. Image object 92 is a house. At this stage, image object 92 blends into the dimly lit forest and is therefore not visible to user U.

[0055] The sensor 20 detects the presence of the user U and the user U's movements.

[0056] When the determination unit 115 determines that the user U is present in the first specific area SA1, the process control unit 111 causes the tactile stimulus processing unit 113 to operate the tactile stimulus device 30. In this example, the tactile stimulus device 30 is a vortex ring generator.

[0057] Figure 5 is a diagram showing the operation of the tactile stimulation device 30. As shown in Figure 5, the tactile stimulation device 30 emits a vortex ring Rv toward the head of the user U. Here, the air inside the tactile stimulation device 30 is heated by a heater, so the temperature of the emitted vortex ring Rv is higher than the ambient temperature. The emitted warm vortex ring Rv travels toward the user U. Because the user U is facing the wall surface 70-1, the vortex ring Rv hits the left side of the user U's head.

[0058] FIG. 6 shows a state in which user U changes the direction of his or her head. When the vortex ring Rv hits the left side of user U's head and user U feels uncomfortable, user U may look around or turn in the direction of the discomfort. In this example, since the vortex ring Rv hits the left side of user U's head, user U turns to the left, i.e., toward wall 70-2, as shown in FIG. 6. At this stage, the brightness of image object 92 may be increased to make the presence of image object 92 more noticeable to user U.

[0059] 7 is a diagram showing how the display mode of the image object 92 changes. As shown in FIG. 7, the image object 92 in the background video 91-2 has a window, and light leaking from the window begins to glow faintly. The change from the image object 92 showing a house with no lights on to the image object 92a showing a house with light leaking from the window occurs when the user U turns toward the wall surface 70-2. In other words, the image object changes from the image object 92 showing a house with no lights on to the image object 92a showing a house with light leaking from the window.

[0060] Fig. 8 is a diagram showing a state in which a user U approaches an image object 92. As shown in Fig. 8, when the user U notices a faint light, he or she approaches the image object 92 to see what it is. That is, the user U moves from the first specific area SA1 to the second specific area SA2.

[0061] 9 is a diagram showing further changes in the display mode of the image object 92. As shown in FIG. 9, when the user U approaches the image object 92, the user U recognizes that the faint light is actually light leaking through a window. When the user U stops in the second specific area SA2, the door 93 of the house opens and a person appears from inside the house. The determination unit 115 can detect that the user U has stopped in the second specific area SA2 by analyzing the photographic data Dt, which is the output of the sensor 20.

[0062] In this way, the user U can have the experience of feeling as if he or she has discovered the image object by noticing it. Note that the display mode in this example, i.e., a mode in which a house stands in a dark forest and light leaking from a window of the house begins to shine, is merely an example, and the present disclosure is not limited to this mode.

[0063] The tactile stimulation device 30 may release the vortex ring Rv when the user U is not standing still in the first specific area SA1. For example, the tactile stimulation device 30 may release the vortex ring Rv when the user U is approaching the tactile stimulation device 30, or when the distance between the user U and the tactile stimulation device 30 is within the reachable distance of the vortex ring Rv.

[0064] 1.5. Specific Operation of Image Display System 1 Next, the operation of the image display system 1, more specifically, the operation of each unit by the processing device 11 in the computer 10 will be described.

[0065] Fig. 10 is a flowchart showing an example of the first processing operation of the processing device 11. Hereinafter, the first processing operation of the processing device 11 will be described with reference to Fig. 10. The routine of Fig. 10 is started, for example, when an instruction to start display is received.

[0066] When an instruction to start display is given, the processing device 11 functions as the processing control section 111 to reproduce the audio-visual data 122 in step S11.

[0067] The processing device 11 transfers the video data of the read video and audio data 122 to the video processing unit 112 and transfers the audio data Ad to the audio processing unit 114. The video processing unit 112 generates video signals Vd-1 and Vd-2 based on the transferred video data and supplies them to the display devices 40-1 and 40-2, respectively. The audio processing unit 114 supplies the transferred audio data Ad to the sound generating device 50.

[0068] The display device 40-1 projects an image based on the video signal Vd-1 onto the wall surface 70-1, and the display device 40-2 projects an image based on the video signal Vd-2 onto the wall surface 70-2. The speakers 52-n to 52-n sequentially generate sounds based on the acoustic signals Ad-1 to Ad-n.

[0069] In step S12, the processing device 11 functions as the processing control unit 111 to acquire the photographing data Dt from the sensor 20.

[0070] In step S13, the processing device 11 functions as the determination unit 115 to analyze the photographing data Dt captured by the sensor 20, and determines whether or not the user U is present in the first specific area SA1 based on the analysis result.

[0071] If it is determined that the user U is present in the first specific area SA1, that is, if the determination result in step S13 is YES, the processing device 11 functions as the tactile stimulus processing unit 113 in step S14, thereby presenting a stimulus to the tactile sense of the user U using the tactile stimulus device 30. In this example, the processing device 11 causes the tactile stimulus device 30 to emit a warm vortex ring Rv toward the user U.

[0072] On the other hand, if it is determined that the user U is not present in the first specific area SA1, that is, if the determination result in step S13 is NO, the processing device 11 functions as the processing control unit 111 in step S12 to acquire the photographed data Dt from the sensor 20. That is, in this case, the processing device 11 executes the processes from step S12 onwards again.

[0073] In step S15, the processing device 11 functions as the processing control unit 111 to acquire the photographing data Dt from the sensor 20.

[0074] In step S16, the processing device 11 functions as the judgment unit 115 to analyze the photographic data Dt captured by the sensor 20, and based on the analysis results, judges whether the first action of the user U satisfies the first condition.

[0075] If it is determined that the first action of the user U satisfies the first condition, that is, if the determination result in step S16 is YES, the processing device 11 displays the first image in the first area in step S17 and temporarily terminates this routine.

[0076] On the other hand, if it is determined that the first action of the user U does not satisfy the first condition, i.e., if the determination result in step S16 is NO, the processing device 11 functions as the tactile stimulation processing unit 113 in step S18 to increase the intensity of the tactile stimulation by one level. This is because there is a possibility that the user U is unaware of the tactile stimulation. Specifically, the processing device 11 changes the setting of the condition parameter for generating the vortex ring Rv by the tactile stimulation device 30 so that the intensity of the tactile stimulation becomes stronger. That is, the processing device 11 presents the stimulation at a first level of intensity, and if it is determined that the first action does not satisfy the first condition, presents the stimulation at a second level of intensity that is stronger than the first level.

[0077] To increase the intensity of the stimulation, for example, the pressure applied to the air inside the tactile stimulation device 30 when releasing the vortex ring Rv can be increased. This increases the size of the released vortex ring Rv. Alternatively, the heating conditions of the heater can be changed to increase the temperature of the air inside the tactile stimulation device 30. Multiple vortex rings Rv can be generated consecutively. Furthermore, fragrances, aroma oils, etc. can be used to impart a scent to the vortex ring Rv.

[0078] After executing the process in step S18, the processing device 11 functions as the tactile stimulus processing unit 113 in step S14, thereby presenting a stimulus to the tactile sense of the user U using the tactile stimulus device 30. That is, in this case, the processing device 11 executes the processes from step S14 onwards again. Step S14 before step S18 is an example of presenting a stimulus at a first degree. Step S14 following step S18 is an example of presenting a stimulus at a second degree.

[0079] Fig. 11 is a flowchart showing an example of a second processing operation that follows a first processing operation of the processing device 11. Hereinafter, the second processing operation of the processing device 11 will be described with reference to Fig. 11. The routine of Fig. 11 is started following the end of the first processing operation.

[0080] When this routine is started following the first processing operation, the processing device 11 functions as the processing control unit 111 to acquire the photographing data Dt from the sensor 20 in step S21.

[0081] In step S22, the processing device 11 functions as the determination unit 115 to analyze the photographic data Dt and determine whether or not the user U is present in the second specific area SA2 based on the analysis result.

[0082] If it is determined that the user U is present in the second specific area SA2, that is, if the determination result in step S22 is YES, the processing device 11 functions as the image processing unit 112 in step S23 to change the first image and temporarily terminate this routine. Specifically, the processing device 11 changes the state of the image object 92a, which is the first image, from a state in which the door is not open to a state in which the door is open.

[0083] On the other hand, if it is determined that the user U is not present in the second specific area SA2, that is, if the determination result in step S22 is NO, the processing device 11 temporarily ends this routine. That is, in this case, the first image does not change.

[0084] 1.6. Advantages of the First Embodiment As described above, the image display method according to the first embodiment includes presenting a stimulus to the tactile sense of user U, determining whether a first action of user U after the stimulus is presented satisfies a first condition based on the output of sensor 20 that detects the action of user U, and, if it is determined that the first action satisfies the first condition, displaying a first image on wall surface 70-2 by display device 40-2.

[0085] According to the image display method described above, the first image is displayed when it is determined that the first action of the user U after the stimulus is presented satisfies the first condition, and the first image is displayed on the wall surface 70-2 in response to a change in the behavior of the user U. This makes the user U feel as if the first image was displayed in response to the user U's own awareness and change in behavior, thereby providing the user U watching the video with a deep sense of immersion and enhancing the sense of realism.

[0086] The fact that the image changes in response to a change in the user U's behavior means that the user U can be immersed in the changes in the image without feeling that the image creator has acted artificially. Therefore, since the user U does not feel that the image creator has acted artificially, they will feel that they have discovered the changes in the image themselves, and we can expect a new interaction between the changes in the image and the user U.

[0087] In addition, in this embodiment, the image changes in response to a change in the behavior of the user U, so the user U concentrates on the image. Therefore, it is expected that this can be applied to indirect treatment or calming down for people with developmental disabilities who tend to be hyperactive or people with mental illnesses who have divergent thinking.

[0088] In the first embodiment, the audiovisual room 70 is an example of a "video space," the wall surface 70-2 is an example of a "first area," and the image object 92 is an example of a "first image" and a "second image." Also, the image object 92a is an example of a "first image" and a "third image." Also, the first image may be displayed on the display device 40-1.

[0089] The first condition is at least one of the following: the user U faces the wall surface 70-2; and the user U views the image displayed in the first area.

[0090] When the user U is made aware by the tactile stimulation, the user U may look around as a result of the awareness, and may therefore turn toward the first area. According to this aspect, the first image is displayed in the first area when the user U turns his / her face toward the first area where the first image will be displayed, making it easier for the user U to find the displayed first image.

[0091] In addition, presenting the stimulus includes at least one of moving the air in the audiovisual room 70 using a blower, bringing a first object in the audiovisual room 70 into contact with the user U using a blower, vibrating a second object that the user U is touching using a vibration device, and bringing a third object in the audiovisual room 70 into contact with the user U using a drive device.

[0092] According to this aspect, the first image is displayed when the user U notices it, making it easier for the user U to find the displayed first image. Note that the string hanging from the ceiling is an example of a "first object," a portion of the floor and the seat of the wheelchair are examples of a "second object," and a portion of the wall is an example of a "third object."

[0093] The blower also includes a vortex ring generator that generates a vortex ring Rv.

[0094] The vortex ring generator generates a vortex ring Rv and moves it through the air, so it can be said to be a device that moves the air in the audiovisual room 70. In this embodiment, it is classified as a type of air blower. According to this aspect, the vortex ring Rv generated by the vortex ring generator can travel a relatively long distance without dissipating, making it easy to target the user U and make the user U aware of it.

[0095] The first action includes at least one of an action of the user U looking around, an action of the user U moving around in the audiovisual room 70, and an action of the user U stopping in a specific area in the audiovisual room 70.

[0096] The user U, who has been made aware by the tactile stimulation, may look around and may move in the direction of the tactile stimulation. Furthermore, the user U, who has been made aware by the tactile stimulation, may stop in place. According to this aspect, the first action is specifically indicated, making it easier to determine whether the first action satisfies the first condition.

[0097] The first image also includes an image that changes from the second image to the third image.

[0098] The image that changes from the second image to the third image is not an image in which an image object is instantly switched to another image object, but an image in which, for example, the brightness, color, pattern, shape, etc. of an image object changes. According to this aspect, the image object changes in response to a change in the behavior of the user U that occurs as a result of presenting a stimulus to the user U's tactile sense, so that even if the degree of change in the image object is slight, the user U can easily notice the change in the image object. Note that an image of a house with the lights off is an example of a "second image," and an image of a house with light spilling out of the windows is an example of a "third image."

[0099] In addition, presenting the stimulus includes presenting the stimulus at a first degree of intensity, and, if it is determined that the first action does not satisfy the first condition, presenting the stimulus at a second degree of intensity that is stronger than the first degree.

[0100] Reactions to tactile stimuli vary from person to person, and if an excessive stimulus is applied to a user U with an overly sensitive sense of touch, the user U will feel uncomfortable. Conversely, if a weak stimulus is applied to a user U with an insensitive sense of touch, the user U may not notice it. For this reason, it is desirable to gradually increase the strength of the tactile stimulus. According to this embodiment, appropriate stimuli can be applied taking into account individual differences in reaction to tactile stimuli.

[0101] Furthermore, the program according to the first embodiment causes a computer to execute determining and displaying. The determining is a process of determining whether a first action of the user U after a stimulus to the user U's tactile sense has satisfied a first condition based on the output of the sensor 20 that detects the action of the user U. The displaying is a process of causing the display device 40-2 to display a first image on the wall surface 70-2 of the audiovisual room 70 when it is determined that the first action has satisfied the first condition.

[0102] According to the above program, the first image is displayed when it is determined that the first action of the user U after the stimulus is presented satisfies the first condition, and the first image is displayed on the wall surface 70-2 in response to a change in the behavior of the user U. This makes the user U feel as if the first image was displayed in response to the user U's own realization or change in behavior, thereby providing the user U watching the video with a deep sense of immersion and enhancing the sense of realism.

[0103] Moreover, the image display system 1 according to the first embodiment includes a sensor 20, display devices 40-1 and 40-2, and a computer 10. The sensor 20 detects the actions of a user U. The display devices 40-1 and 40-2 display images in an audiovisual room 70. The computer 10 executes the processes of determining and displaying. The determining process is a process of determining, based on the output of the sensor 20, whether a first action of the user U after a stimulus to the user U's tactile sense has satisfied a first condition. The displaying process is a process of causing the display device 40-2 to display a first image on a wall surface 70-2 of the audiovisual room 70 when it is determined that the first action has satisfied the first condition.

[0104] According to the image display system 1 described above, the first image is displayed when it is determined that the first action of the user U after the presentation of the stimulus satisfies the first condition, and the first image is displayed on the wall surface 70-2 in response to a change in the behavior of the user U. This makes the user U feel as if the first image was displayed in response to the user U's own awareness and change in behavior, thereby providing the user U watching the video with a deep sense of immersion and enhancing the sense of realism.

[0105] 2. Second embodiment An image display system 1A according to a second embodiment of the present invention will be described below with reference to Figures 12 to 14. In the following description, for the sake of simplicity, the same components as those in the first embodiment will be denoted by the same reference numerals, and a description of their functions may be omitted. In addition, in the following description, for the sake of simplicity, differences between the second embodiment and the first embodiment will be mainly described.

[0106] 2.1. Configuration of the Second Embodiment

[0107] 12 is a configuration diagram of an image display system 1A according to the second embodiment. The image display system 1A includes a computer 10A, a sensor 20, a tactile stimulation device 30, and MR (Mixed Reality) glasses 40A.

[0108] In the first embodiment, the image space is a real space, but in the second embodiment, the image space is a virtual space or a mixed reality space that combines a real space and a virtual space. By wearing the MR glasses 40A on the head, the user U can view images and sounds displayed in the mixed reality space.

[0109] The computer 10A differs from the computer 10 according to the first embodiment in that the computer 10A displays images on the MR glasses 40A and outputs sounds. The configurations of the sensor 20 and the tactile stimulation device 30 according to the second embodiment are the same as the configurations of the sensor 20 and the tactile stimulation device 30 according to the first embodiment.

[0110] The computer 10A controls each unit in the image display system 1A. The computer 10A includes a processing device 11A, a storage device 12A, an input IF 13, and an output IF 14. These devices are electrically connected to each other via, for example, a bus Bs.

[0111] The processing device 11A controls the operation of the computer 10A. The processing device 11A includes one or more CPUs. However, the processing device 11A may include an FPGA, a DSP, an ASIC, etc. instead of or in addition to a CPU.

[0112] The storage device 12A is configured to include, for example, a volatile memory such as RAM and a non-volatile memory such as ROM. The storage device 12A stores a control program 121 for the processing device 11A to control each unit, video and audio data 122A consisting of video data and audio data, and control data 123 including control parameters for the tactile stimulation device 30. The volatile memory of the storage device 12A is used by the processing device 11A as a work area for the processing device 11A.

[0113] The configurations of the input IF13, output IF14, sensor 20, and tactile stimulation device 30 are similar to the configurations of the input IF13, output IF14, sensor 20, and tactile stimulation device 30 in the first embodiment, so detailed explanations will be omitted.

[0114] The MR glasses 40A are a wearable display device worn on the head of a user U. The MR glasses 40A display virtual image objects on display panels provided on lenses corresponding to both eyes of the user U. Each lens and display panel of the MR glasses 40A is see-through. Therefore, the user U wearing the MR glasses 40A can view the real space as an external world image through each lens and display panel of the MR glasses 40A. The external world image may be a virtual image obtained by capturing an image of the user U's surroundings. A method using a real external world image is called an optical see-through method. A method using a virtual external world image is called a video see-through method. In the following explanation, the MR glasses 40A will be described as employing the optical see-through method. The MR glasses 40A are an example of a display device.

[0115] 2.2. Functions of the Processing Unit Fig. 13 is a functional block diagram showing an example of the functions of a processing device 11A according to the second embodiment. Fig. 13 shows a functional configuration constructed when the processing device 11A of the computer 10A executes a control program 121A stored in the storage device 12A. As shown in Fig. 13, the processing device 11A has functions as a processing control unit 111A, a video processing unit 112A, a tactile stimulation processing unit 113, a sound processing unit 114, and a determination unit 115.

[0116] The processing control section 111A starts reading out the video and audio data 122 from the storage device 12A by a predetermined operation or the like, and controls the video processing section 112A, tactile stimulus processing section 113, audio processing section 114, and determination section 115.

[0117] The video processing unit 112A generates a video signal Vd based on the video data of the video and audio data 122 read by the processing control unit 111A under the control of the processing control unit 111. The video signal Vd generated in the video processing unit 112A is supplied to the MR glasses 40A via the output IF14.

[0118] The tactile stimulus processing unit 113 generates a control signal Cd based on the control data 123 read by the processing control unit 111A under the control of the processing control unit 111A. The control signal Cd generated in the tactile stimulus processing unit 113 is supplied to the tactile stimulus device 30 via the output IF14.

[0119] The audio processing unit 114 outputs audio data Ad from the audio-visual data 122 read by the processing control unit 111A. The audio data Ad output from the audio processing unit 114 is supplied to the MR glasses 40A via the output IF14.

[0120] The determining unit 115 has the same function as the determining unit 115 in the first embodiment, and therefore a description thereof will be omitted.

[0121] 2.3. MR Glass 40A Configuration Example Fig. 14 is a perspective view showing the appearance of the MR glasses 40A of Fig. 13. The MR glasses 40A include temples 94L and 94R, a bridge 96, projection optical systems 98L and 98R, and a sound output device 53. In the following description, when distinguishing between similar elements, a suffix such as "L" for temple 94L and "R" for temple 94R is used. When not distinguishing between similar elements, only a common number without a suffix is ​​used, such as temple 94.

[0122] The temples 94 are rod-shaped components supported by the pinna. The bridge 96 is disposed between the projection optical system 98L and the projection optical system 98R. The projection optical system 98 includes a display device 37, a light guide path 981, and a half mirror 982. The bridge 96 is provided with the imaging device 35. The imaging device 35 captures an image of the external world in front of the user U.

[0123] The display device 37 is disposed within the temple 94. The display device 37 displays an image. The display device 37 has various display panels, such as a liquid crystal panel and an organic EL (Electro Luminescence) panel. When the display device displays an image, light representing the image is emitted from the display device 37. The light emitted from the display device 37 is guided to a half mirror 982 by a light guide path 981. The half mirror 982 reflects the light guided by the light guide path 981. The light reflected by the half mirror 982 is projected onto the retina of the user U. The user U recognizes the image using this light. The half mirror 982 has a surface facing the user U. When the display device 37 does not display an image, the user U can see the outside world through the half mirror 982.

[0124] The sound output device 53 is disposed on the side of the temple 94. The sound output device 53 outputs sound. In this embodiment, the MR glasses 40A are controlled by the computer 10A and therefore do not have any particular input device, but may be provided with an input device such as a touch sensor or a sound collection device such as a microphone. If the MR glasses 40A have an acceleration sensor built in, it may be used as one of the sensors 20 that detects the movement of the user U's head.

[0125] 2.4. Example of Image Display System 1A Layout Fig. 15 is a diagram showing an example of the arrangement of each element in an image display system 1A according to the second embodiment. Note that each element in the image display system 1A is provided in an audiovisual room 70, but in Fig. 15, part of the ceiling and walls of the audiovisual room 70 are omitted to make the arrangement example easier to understand. The sensor 20 and the tactile stimulation device 30 are attached to a wall surface 70-2, for example. The MR glasses 40A are worn on the head of a user U.

[0126] The MR glasses 40A are provided with a sound output device 53, and sound can be output from the sound generating device 50, but the sound may also be output from earphones worn by the user U. Also, a speaker 52-1 may be provided in the audiovisual room 70, and sound may be output from the speaker 52-1 as in the first embodiment.

[0127] 2.5. Operation of Image Display System 1A Attitude information of the MR glasses 40A can be acquired by analyzing output signals Acc from various sensors including an acceleration sensor built into the MR glasses 40A. Also, three-dimensional position information of the MR glasses 40A can be acquired by analyzing imaging data Dta from the imaging device 35 of the MR glasses 40A. Three-dimensional position information of the MR glasses 40A can also be acquired, for example, by inertial navigation calculation using output signals Acc from various sensors such as acceleration sensors, starting from the entrance of the audiovisual room 70.

[0128] The processing device 11A renders the video signal Vd based on the analyzed rotation information and three-dimensional position information of the MR glasses 40A. The processing device 11A supplies the rendered video signal Vdr to the MR glasses 40A. As a result, an image that changes depending on the rotation direction and three-dimensional position of the MR glasses 40A is displayed on the display device 37 of the MR glasses 40A. Therefore, the processing device 11A can display the image on the display device 37 as if it were projected onto the wall surfaces 70-1 and 70-2.

[0129] 2.6. Advantages of the Second Embodiment As described above, the image display system 1A according to the second embodiment includes the sensor 20, the MR glasses 40A, and the computer 10A. The sensor 20 detects the movement of the user U. The MR glasses 40A display an image in a mixed reality space. The computer 10A executes the processes of determining and displaying. The process of determining is a process of determining, based on the output of the sensor 20, whether a first movement of the user U after presenting a stimulus to the user U's tactile sense satisfies a first condition. The process of displaying is a process of causing the MR glasses 40A to display a first image in a first area of ​​the mixed reality space when it is determined that the first movement satisfies the first condition.

[0130] According to this aspect, by using the MR glasses 40A, the user U can view mixed reality in which the wall surfaces 70-1 and 70-2 in the real space are superimposed on the background image and image objects in the virtual space. Therefore, the user U can have an experience similar to that of the user U described in the first embodiment. Furthermore, according to this aspect, when it is determined that the first action of the user U satisfies the first condition, the first image is displayed on the display device 37. Therefore, the first image is displayed based on the awareness and behavior of the user U, so that the user U can be immersed in viewing the image.

[0131] 3. Variations The present invention is not limited to the above-described embodiment, and various modifications can be adopted within the scope of the present invention. Specific modified embodiments are exemplified below. Furthermore, two or more embodiments arbitrarily selected from the following examples may be appropriately combined within a range that does not contradict each other. In the modified embodiments exemplified below, for elements whose actions and functions are equivalent to those of the above-described embodiment, the symbols used in the above explanation will be used, and detailed explanations of each will be omitted as appropriate.

[0132] 3.1. Variation 1 In the above embodiments, the timing of providing the tactile stimulation is when the user is in the first specific area, but it may also be according to a predetermined timetable, synchronized with the video data, or after a certain period of time has elapsed, or may be random.

[0133] 3.2. Variation 2 In the above embodiments, the appearance of the image object superimposed on the background video is changed in response to a change in the user's behavior upon noticing a tactile stimulus, but the displayed image object may be instantaneously switched to another image object, or the background video may be dynamically switched to another background video.

[0134] 3.3. Variation 3 In each of the above embodiments, it is assumed that there is only one user in the audiovisual room 70, but if there are two or more users, the first image may be displayed when, for example, the number of users whose first behavior satisfies the first condition exceeds the majority of all users present.

[0135] 3.4. Variation 4 In the above embodiments, the image display is changed in response to a change in the user's behavior that occurs as a result of stimulating the user's sense of touch. However, the stimuli given to the user may include a stimulus to a sense other than touch, such as a stimulus to the sense of sight. Specifically, an image object may be displayed in the image to alert the user, or an image different from that displayed in other areas may be displayed in one area.

[0136] The sense of hearing may also be used as a stimulus to the user. For example, while a user is watching a video accompanied by background music, a sound different from the background music may be generated at a certain timing, and a change in the user's behavior indicating that the user has noticed the sound may be used to change the appearance of the video that has been displayed up to that point. The sense of smell may also be used as a stimulus to the user. For example, a fragrant liquid may be sprayed, and a change in the appearance of the video may be used to change the appearance of the video when the user has noticed the scent.

[0137] 3.5. Variation 5 In the first embodiment, a device that can be used as the tactile stimulation device 30 is exemplified, but the tactile stimulation device 30 is not limited to the device exemplified in the first embodiment. For example, the tactile stimulation device 30 may be a dedicated device that reproduces the situation in which a dead branch breaks when stepped on. The dedicated device has, for example, a pressure sensor, an actuator, and an aluminum plate that has been deformed into an uneven shape, and is placed under the floor surface. When stepped on by a user U, the pressure sensor outputs a signal corresponding to the pressure of the stepping. When the signal output from the pressure sensor exceeds a threshold, the actuator generates stress in the aluminum plate, causing it to distort. By stepping on such a dedicated device, the user U's tactile sense is stimulated, and the user U may become aware of something.

[0138] 3.6. Variation 6 In the second embodiment, a case has been described in which the user U uses the MR glasses 40A to view mixed reality in which the wall surfaces 70-1 and 70-2 in the real space are superimposed on the background video and image objects in the virtual space. However, the present invention is not limited to the case of the second embodiment, and may be a case in which the user U uses the MR glasses 40A to view a video in which the background video and image objects are projected onto a screen in the virtual space.

[0139] 3.7. Variation 7 The MR glasses 40A according to the second embodiment are a see-through type device that allows a user to experience mixed reality. However, instead of the MR glasses 40A, virtual reality (VR) goggles that allow a user to experience virtual reality may be used. When VR goggles are used, a user U wearing the VR goggles on their head basically stays at a predetermined viewing position to view the video. However, even in this mode, the first image is displayed in the virtual space due to the user U's own awareness and actions, so it is possible to immerse the user U in viewing the video.

[0140] 4. Notes A summary of this disclosure is provided below.

[0141] Appendix 1 The image display method according to Appendix 1 includes presenting a stimulus to a user's tactile sense, determining whether a first action of the user after presenting the stimulus satisfies a first condition based on the output of at least one sensor that detects the user's action, and, if it is determined that the first action satisfies the first condition, displaying a first image in a first region of a video space by at least one display device.

[0142] According to the image display method of Supplementary Note 1, when it is determined that the first action of the user after the presentation of the stimulus satisfies the first condition, the first image is displayed. Since the first image is displayed in response to a change in the user's behavior, the user is less likely to sense that the video producer has acted artificially, and is more likely to sense that the video has changed in response to the user's own awareness and change in behavior, which can provide a deep sense of immersion to the user watching the video.

[0143] Appendix 2 The image display method according to Supplementary Note 2 is the same as that according to Supplementary Note 1, wherein the first condition is at least one of the user facing the user's face in the direction of the first area and the user viewing the image displayed in the first area.

[0144] According to the image display method according to Supplementary Note 2, the user can easily find the displayed first image.

[0145] Appendix 3 The image display method according to Appendix 3 is in Appendix 1 or Appendix 2, wherein presenting the stimulus includes at least one of moving air in the image space with a blower, bringing a first object in the image space into contact with the user with the blower, vibrating a second object that the user is touching with a vibration device, and bringing a third object in the image space into contact with the user with a drive device.

[0146] According to the image display method of Supplementary Note 3, similarly to Supplementary Note 2, the user can easily find the displayed first image.

[0147] Appendix 4 The image display method according to Supplementary Note 4 is the method according to Supplementary Note 3, wherein the blower includes a vortex ring generator that generates a vortex ring.

[0148] According to the image display method according to Supplementary Note 4, it is easy to target the user and to give the user an awareness.

[0149] Appendix 5 The image display method according to Supplementary Note 5 is any one of Supplementary Note 1 to Supplementary Note 4, wherein the first action includes at least one of the following actions: the user looking around, the user moving within the image space, and the user stopping in a specific area within the image space.

[0150] According to the image display method of Supplementary Note 5, the first action is specifically displayed, making it easier to determine whether or not the first action satisfies the first condition.

[0151] Appendix 6 The image display method according to Supplementary Note 6 is any one of Supplementary Note 1 to Supplementary Note 5, wherein the first image includes an image that changes from a second image to a third image.

[0152] According to the image display method of Appendix 6, the image object changes in response to a change in the behavior of user U that occurs as a result of stimulating the user's tactile sense, so that even if the degree of change in the image object is slight, the user can easily notice the change in the image object.

[0153] Appendix 7 The image display method according to Appendix 7 is any one of Appendixes 1 to 6, wherein presenting the stimulus includes presenting the stimulus at a first degree of intensity, and, if it is determined that the first action does not satisfy the first condition, presenting the stimulus at a second degree that is stronger than the first degree.

[0154] According to the image display method of Supplementary Note 7, it is possible to provide an appropriate stimulus in consideration of individual differences in response to tactile stimuli.

[0155] Appendix 8 The program according to Appendix 8 causes a computer to determine whether a first action of a user after presenting a stimulus to the user's tactile sense satisfies a first condition based on the output of at least one sensor that detects the user's action, and, if it is determined that the first action satisfies the first condition, cause at least one display device to display a first image in a first region of a video space.

[0156] According to the program of Appendix 8, as with Appendix 1, it is difficult to sense the video creator's artificiality, and it is easier to sense that the video has changed as a result of the user's own awareness and changes in behavior, thereby giving the user watching the video a deep sense of immersion.

[0157] Appendix 9 The image display system of Appendix 9 includes at least one sensor that detects a user's movement, at least one display device that displays an image in a video space, and an information processing device that performs the following operations: determining whether a first movement of the user after presenting a stimulus to the user's tactile sense satisfies a first condition based on the output of the at least one sensor; and, when it is determined that the first movement satisfies the first condition, causing the at least one display device to display a first image in a first area of ​​the video space.

[0158] According to the image display system of Appendix 9, similar to Appendix 1, it is difficult to sense the video producer's artificiality, and it is easier to sense that the video has changed in response to the user's own awareness and changes in behavior, thereby providing a deep sense of immersion to the user watching the video. [Explanation of symbols]

[0159] 1,1A...image display system, 10,10A...computer, 11...processing device, 20...sensor, 30...tactile stimulation device, 40-1,40-2...display device, Rv...vortex ring, U...user.

Claims

1. presenting a stimulus to a user's sense of touch; determining whether a first action of the user after the stimulus is presented satisfies a first condition based on an output of at least one sensor that detects the action of the user; When it is determined that the first action satisfies the first condition, displaying a first image in a first region of a video space by at least one display device; An image display method including:

2. the first condition is at least one of the user facing the first area and the user viewing an image displayed in the first area; The image display method according to claim 1 .

3. Presenting the stimulus comprises: moving air within the image space with a blower; causing the air blower to bring a first object in the image space into contact with the user; vibrating a second object touched by the user using a vibration device; causing a third object in the image space to come into contact with the user by a drive device; 3. The image display method according to claim 1, further comprising at least one of the steps of:

4. The blower includes a vortex generator that generates a vortex ring. The image display method according to claim 3.

5. The first action includes at least one of an action of the user looking around, an action of the user moving within the image space, and an action of the user stopping in a specific area within the image space.

3. The image display method according to claim 1 or 2.

6. The first image includes an image that changes from a second image to a third image.

3. The image display method according to claim 1 or 2.

7. Presenting the stimulus comprises: presenting the stimulus at a first intensity; and when it is determined that the first action does not satisfy the first condition, presenting the stimulus at a second intensity that is stronger than the first intensity.

3. The image display method according to claim 1 or 2.

8. determining whether a first action of the user after presenting a stimulus to the user's tactile sense satisfies a first condition based on an output of at least one sensor that detects the user's action; When it is determined that the first action satisfies the first condition, causing at least one display device to display a first image in a first region of a video space; to have the computer run program.

9. at least one sensor for detecting user motion; at least one display device that displays an image in a video space; determining whether a first action of the user after presenting a stimulus to the user's tactile sense satisfies a first condition based on an output of the at least one sensor; when it is determined that the first action satisfies the first condition, causing the at least one display device to display a first image in a first region of the video space; an information processing device that executes the above; An image display system comprising:

Images